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From  the  Publisher 


Smooth  Sailing  for  Application  Security 


So  many  of  an  organization’s  business  practices  are  now  pushed  to  the  Web.  In-house 
and  external  applications  are  extending  secure  data  and  proprietary  information  out 
into  a  pool  of  expanded  users  in  order  to  satisfy  business  needs.  The  dangers  of  this 
practice  include  other  pool  dwellers  that  might  be  malicious  in  nature,  seeking  to  exploit 
what  so  many  software  professionals  and  system  users  hold  dear  —  information. 

The  nature  of  this  issue’s  focus,  Application  Security,  reminds  me  of  the  story  of  Dame 
EUen  MacArthur,  who  completed  the  fastest  solo  non-stop  circumnavigation  of  the 


globe  in  less  than  72  days.  MacArthur  courageously  sailed  dangerous  waters,  surrounded  by  ice¬ 
bergs,  massive  swells,  and  sea-dweUing  predators  similar  to  the  shark  on  this  month’s  cover  (but 
without  the  protective  capabilities  of  the  porcupine  fish).  Like  MacArthur  in  her  sailboat,  mak¬ 
ing  her  way  through  a  treacherous  sea,  precious  organizational  information  flows  through  appli¬ 
cation  software  out  to  a  community  of  users,  vendors,  and  customers,  with  malicious  hackers 
always  present.  This  is  risky  business  indeed. 

Joe  Jarzombek,  Director  for  Software  Assurance  for  the  National  Cyber  Security  Division  of 
the  Department  of  Homeland  Security  (DHS),  while  addressing  an  Advanced  Software 
Acquisition  Management  class  at  the  Defense  Acquisition  University,  held  up  a  copy  of  the  June 
2008  issue  of  CROSSTALK  on  Software  Quality  as  a  useful  source  and  said,  “Rather  than 
attempt  to  break  or  defeat  network  or  systems  security,  hackers  are  opting  to  target  application 
software  to  circumvent  security  controls.”  The  numbers  bear  that  out  as  Jarzombek  noted  that 
Gartner,  Inc.,  an  information  technology  research  and  advisory  company,  found  that  90  percent 
of  software  attacks  were  aimed  at  the  application  layer.  Jarzombek  also  believes  that  “most 
exploitable  software  vulnerabilities  are  attributable  to  non-secure  coding  practices  and  not  iden¬ 
tified  in  testing.”  He  asserted,  “Functional  correcmess  must  be  exhibited  even  when  software  is 
subjected  to  abnormal  and  hostile  conditions.” 

So  what  is  a  software  professional  to  do?  This  month’s  articles  provide  an  application  secu¬ 
rity  lifeline.  The  lead  article.  Securing  Legacy  C  Applications  Using  Dynamic  Data  Flow  Analysis  by 
Steve  Cook,  Dr.  Calvin  Lin,  and  Walter  Chang  offers  solutions  in  securing  existing  code  within 
applications.  Building  Secure  Systems  Using  Model-Based  Engineering  and  Architectural  Models  by  Dr. 
Jorgen  Hansson,  Dr.  Peter  H.  Feiler,  and  John  Morley,  and  Practical  Defense  in  Depth  by  Michael 
Howard  offer  structural  support  to  anyone  looking  to  bolster  their  security  practices  on  future 
designs.  Likewise,  Karen  Mercedes  Goertzel’s  article.  Enhancing  the  Software  Development  Life  Cycle 
[SDLC]  to  Produce  Secure  Software,  shares  the  DHS  Software  Assurance  Program’s  perspective  of 
what  makes  software  secure,  while  leaving  specific  SDLC  security-development  up  to  the  read¬ 
er.  In  Supporting  Safe  Content-Inspection  of  Web  Traffic,  Dr.  Partha  Pal  and  Michael  Atighetchi  show 
how  Hypertext  Transfer  Protocol  Secure  (HTTPS)  proxies  enable  safe  interception  and  inspec¬ 
tion  of  HTTPS  traffic,  while  Corey  P.  Cunha’s  article  Hazardous  Software  Development  explores 
past  safety-critical  systems  failures  and  modern  software  solutions.  And  finally,  this  issue’s  co¬ 
sponsor,  the  DHS  Software  Assurance  Program,  offers  many  free  resources  for  Application 
Security  —  just  follow  the  link  on  the  back  cover. 

So,  to  all  those  charged  with  the  difficult  duty  of  securing  your  organizations  applications, 
remember  the  promising  success  of  Dame  EUen  MacArthur.  She,  just  Uke  you,  set  sail  into  dan¬ 
gerous  waters,  yet  she  arrived  safely  and  without  a  single  scratch.  To  aU  our  CrossTalk  read¬ 
ers:  I  hope  this  issue  helps  in  navigating  through  the  sea  of  threats  and  leads  to  smooth  saiUng 
in  aU  of  your  security  efforts. 


Kasey  Thompson 
Publisher 
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Securing  Legacy  C  Applications  Using 
Dynamic  Data  Flow  Analysis 


Steve  Cook  Dr.  Calvin  Lin  and  Walter  Chang 

Southwest  Research  Institute  University  of  Texas  at  Austin 

Most  attacks  on  networks  and  information  systems  consist  of  exploits  of  software  vulnerabilities.  Even  safe  programming 
languages  are  not  immune  to  problems  such  as  Structured  Qttery  Eanguage  (SQIS)  injection,  cross-site  scripting,  or  other 
information  flow-related  vulnerabilities.  While  current  technological  and  educational  efforts  seek  to  ensure  the  security  of 
future  applications,  millions  of  lines  of  existing  software  must  be  secured  as  we  work  to  defend  our  national  infrastructure. 

What  is  needed  is  an  automatic  and  scalable  method  that  identifies  and  traps  runtime  exploits  and  that  can  update  existing 
software  as  security  policies  evolve.  This  article  presents  ongoing  research  on  an  approach  to  securing  legacy  C  applications, 
while  remaining  applicable  to  future  problems  and  languages. 


Most  attacks  on  networks  and  infor¬ 
mation  systems  begin  by  exploiting 
a  vulnerability  in  a  software  application 
that  is  resident  on  a  host  computer,  serv¬ 
er,  or  even  an  appliance  designed  to  pro¬ 
vide  network  defense.  Addressing  these 
vulnerabilities  is  currently  very  labor- 
intensive,  requiring  constant  updates  and 
patches.  AH  of  us  have  become  accus¬ 
tomed  to  receiving  software  security 
updates  on  an  almost  daily  basis  for  many 
of  our  commonly  used  applications. 
Terms  such  as  denial  of  service,  phishing, 
botnets,  and  spamming  are  all  becoming 
part  of  our  everyday  vernacular,  and  our 
collective  concern. 

Current  technological  and  educational 
efforts  seek  to  ensure  the  security  of 
future  applications.  Most  approaches  to 
resolving  today’s  security  concerns  focus 
on  single-point  solutions.  Furthermore, 
millions  of  lines  of  existing  software  that 
comprise  our  legacy  systems  must  be 
secured  to  defend  those  information  sys¬ 
tems  on  which  our  national  infrastructure 
depends.  Unfortunately,  today’s  software 
engineers  are  not  typically  trained  in  the 
development  of  secure  software  systems. 
Implementation  of  application  security 
requires  that  the  programmer  be  an 

Figure  1:  Overall  Architecture  of  DDFA  System 


expert  not  only  in  the  application 
domain,  but  also  in  secure  coding  prac¬ 
tices.  Our  universities  are  beginning  to 
add  security  to  the  software  educational 
curriculum  so  that  new  graduates  can  dis¬ 
tinguish  good  practices  from  bad  as  they 
translate  software  designs  into  source 
code.  The  Department  of  Flomeland 
Security  has  created  a  publicly  available 
Web  site  to  capture  best  practices  in 
developing  secure  software  at  <https:// 
buildsecurityin.us-cert.gov> .  Flowever, 
even  when  a  programmer  is  trained  in 
best  practices  for  secure  programming,  it 
is  unrealistic  to  depend  upon  the  pro¬ 
grammer  to  develop  code  that  is  absent 
of  vulnerabilities. 

To  supplant  the  ad-hoc  security 
enhancing  efforts  of  today  and  meet  the 
challenges  of  tomorrow,  an  automated 
method  is  needed  to  identify  and  guard 
against  runtime  exploits  while  remaining 
flexible  and  agile  as  security  policies 
change  and  evolve.  In  this  article,  we 
present  research  work  in  progress  to 
develop  a  system  that  ensures  that  C 
programs  enforce  a  wide  variety  of  user- 
defined  security  policies  with  a  mini¬ 
mum  of  runtime  overhead  and  disrup¬ 
tion  to  development  processes.  In  the 


future,  our  system  can  be  extended  to 
handle  multiple  languages  and  comple¬ 
ment  new  security  solutions. 

What  Is  Dynamic  Data  Flow 
Analysis  (DDFA)? 

DDFA  is  an  extensible,  compiler-based 
system  that  automatically  instruments  the 
source  code  of  arbitrary  (meaning  without 
any  assumptions  on  the  code)  C  programs 
to  enforce  a  user-specified  security  policy. 
The  system  does  not  require  any  modifi¬ 
cation  to  the  original  source  code  by  the 
developer  and  also  does  not  significantly 
degrade  a  program’s  runtime  perfor¬ 
mance.  Moreover,  it  has  the  ability  to 
simultaneously  enforce  many  different 
classes  of  security  vulnerabilities. 

The  DDFA  system  is  built  upon  the 
Broadway  static  data  flow  analysis  and 
error  checking  system,  which  is  a  source- 
to-source  translator  for  C  developed  by 
the  computer  sciences  department  at  the 
University  of  Texas  at  Austin  (UT-Austin) 
[1].  UT-Austin  and  the  Southwest  Re¬ 
search  Institute  (SwRI)  are  collaborating 
to  enhance  the  Broadway  specification 
language  and  analysis  infrastructure  with  a 
dependence  analysis,  an  instrumentation 
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Traditional  Tainted  Data  Attacks 

Security  Problems  Taint  Tracking 
Cannot  Handle 

Format  String  Attacks 

File  Disclosure  Vulnerabilities 

SQL  Injection 

Labeled  Security  Enforcement 

Command  Injection 

Role-Based  Access  Control 

Cross-Site  Scripting 

Mandatory  Access  Control 

Privilege  Escalation 

Accountable  Information  Flow 

Table  1:  Vulnerabilities  That  Taint  Tracking  Can  and  Cannot  Handle 


engine,  and  a  dynamic  data  flow  library. 

Figure  1  shows  the  overall  architecture 
of  the  DDFA  system.  Input  to  the 
Broadway  compiler  consists  of  the  source 
code  of  an  untrusted  program  and  a  secu¬ 
rity  policy  specification  file.  The  output  is 
an  enhanced  version  of  the  original  source 
code  that  has  been  automatically  instru¬ 
mented  with  DDFA  runtime  library  calls. 
The  modified  program  is  then  compiled 
for  the  platform  of  choice  so  that  its  secu¬ 
rity  policy  can  be  enforced  at  runtime 
using  DDFA.  The  system  does  not  require 
hardware  or  operating  system  changes. 

The  primary  design  goals  are  the  fol¬ 
lowing: 

1.  Minimize  the  required  developer 
involvement. 

2.  Minimize  runtime  overhead  of  the 
secured  program. 

3.  Provide  sufficient  generality  for  multi¬ 
level  security  support  and  enough 
extensibility  for  future  capabilities. 
Minimizing  developer  involvement  is 

achieved  through  a  security  policy  speci¬ 
fication  file  that  is  independent  of  the 
program.  A  security  policy  is  defined 
once  by  a  security  expert  using  a  simple 
language,  which  has  a  direct  mapping  to 
the  application  programming  interface  to 
which  the  program  is  written.  The  policy, 
once  defined,  can  be  applied  to  many  dif¬ 
ferent  programs.  The  DDFA  approach  is 
easily  integrated  into  the  development 
workflow,  adding  only  an  additional 
compilation  step  before  application 
deployment. 

To  minimize  the  runtime  overhead  of 
an  executing  program,  the  DDFA 
approach  builds  on  the  body  of  research 
in  static  analysis  and  leverages  semantic 
information  provided  by  the  security  pol¬ 
icy  to  enable  optimizations  beyond  stan¬ 
dard  compiler  techniques.  This  results  in 
a  program  that  is  instrumented  with  addi¬ 
tional  code  only  where  provablj  necessary, 
so  innocuous  flows  of  data  are  not 
tracked  at  runtime,  thus  keeping  runtime 
overhead  low. 

For  sufficient  generality  and  extensi¬ 
bility  in  security  exploits,  a  DDFA 
approach  is  used  instead  of  a  more  tradi¬ 
tional  dynamic  taint  analysis  [2,  3,  4]. 
Taint  analysis  tracks  the  flow  of  tainted 
data  (i.e.,  data  originating  from  the 
potential  attacker)  through  the  system  at 
runtime  and  then  checks  that  the  tainted 
data  is  not  misused.  Our  approach 
expands  on  the  generality  of  taint  track- 
ing  by  recognizing  that  taint  tracking  is  a 
special  case  of  data  flow  tracking 
(DDFA).  Whereas  taint  analysis  typically 
tracks  one  bit  of  information,  data  flow 
analysis  can  track  multiple  bits  of  infor¬ 


mation  and  can  combine  the  information 
in  more  flexible  ways  than  taint  analysis. 
This  allows  our  approach  to  support 
multi-level  security  and  provide  a  higher 
level  of  generality  that  could  be  used  for 
other  unanticipated  security  challenges  in 
the  future. 

Table  1  identifies  exploits  that  can 
and  cannot  be  handled  by  taint-based  sys¬ 
tems.  Since  the  DDFA  approach  sup- 

supplant  the  ad-hoc 
security-enhancing  efforts 
of  today  and  meet  the 
challenges  of  tomorrow, 
on  automated  method  is 
needed  to  identify  and 
guard  against  runtime 
exploits  while  remaining 
flexible  and  agile  as 
security  policies  change 
and  evolve/* 


ports  general  data  flow  tracking,  it  can 
handle  all  of  these  exploits,  and  can  do  so 
simultaneously.  Such  generality  will 
become  particularly  important  as  devel¬ 
opers  move  to  memory-safe  languages 
such  as  Java  and  C#,  where  the  use  of 
taint  tracking  to  enforce  secure  control 
flow  is  not  as  important.  Security  vulner¬ 
abilities  that  are  not  currently  addressed 
by  the  DDFA  approach  include  those 
such  as  covert  timing  channel  vulnerabil¬ 
ities,  since  they  are  outside  the  scope  of 
data  flow  analysis  [5]. 

How  Does  DDFA  Work? 

In  this  section,  we  will  discuss  in  more 
detail  the  components  of  the  DDFA  sys¬ 


tem  and  how  they  work.  The  components 
include  the  following:  the  poliy  specification, 
static  data  flow  analysis,  instrumentation  engine, 
and  DDFA. 

Policy  Specification 

The  policy  specification  is  defined  using 
the  Broadway  specification  language  [1, 
6] .  It  is  a  simple  declarative  language  that 
is  used  to  teU  the  compiler  how  to  per¬ 
form  specific  data  flow  analysis  by  sup- 
plying  rules.  The  two  high-level  items  to 
specify  are  the  lattice  definition  and  the 
summaries  for  the  library  functions  and 
system  calls. 

A  lattice  must  be  defined  for  each  type 
of  analysis  to  be  performed.  For  example, 
in  a  format  string  attack,  taint  analysis  can 
be  used.  A  lattice  called  Taint  would  be 
defined  with  two  flow  values  which  could 
be  named  Tainted  and  Untainted.  Further¬ 
more,  Untainted  is  placed  higher  than 
Tainted  to  signify  that  when  the  two  flow 
values  are  combined  on  a  particular  object 
being  tracked,  the  result  is  the  lower  of  the 
two.  Tainted.  Lattices  naturally  model  hier¬ 
archical  security  levels  and  are  ideal  for 
reasoning  about  multilevel  security  and 
access  control  that  go  beyond  taint  track¬ 
ing,  such  as  role-based  access  control 
(RBAC)  [7]. 

Summaries  for  the  library  functions 
and  system  calls  define  how  the  lattice 
flow  values  are  introduced  into  the  sys¬ 
tem,  how  the  flow  values  propagate 
through  the  system,  and  how  to  track 
unsafe  use  of  the  lattice  flow  values. 
Continuing  our  format  string  attack  exam¬ 
ple,  the  following  specification  declares 
that  data  introduced  into  the  system 
through  the  network  library  call  recvQ 
would  always  be  tainted: 

procedure  recv(s,  buf,  len,  flags) 

{ 

on  entry  {  buf  ^  buffer } 
analyze  Taint  {  buffer  ^  Tainted  } 

} 

Here,  the  on_entry  keyword  describes  func¬ 
tion  parameters  relevant  to  the  analysis 
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and  gives  a  name,  buffer,  to  the  object 
pointed  to  by  buf.  The  analyse  keyword 
describes  the  effect  of  the  function  on  lat¬ 
tice  flow  values. 

To  allow  the  compiler  to  reason  about 
the  propagation  of  tainted  data,  the  fol¬ 
lowing  specification  of  the  library  system 
call  strdupQ  declares  that  it  returns  a  point¬ 
er  to  string^copj  and  that  strinp^copy  should 
have  whatever  taintedness  that  string  has: 

procedure  strdup(s) 

{ 

on  entry  { s  string  } 
on  exit  { return  string  copy  } 
analyze  Taint  { string  copy  «-  string  } 

} 

Finally,  to  track  the  unsafe  use  of  tainted 
data,  an  error  handler  would  be  specified 
as  follows  in  the  summary  for  a  library  call 
such  as  prinfO  (an  ultimate  perpetrator  in  a 
format  string  attack)  when  unsafe  data  is 
actually  used: 

procedure  printf(format,  args) 

{ 

on  entry  { format  ^  format_string  } 
errorjf  (Taint :  format_string  could-be 
Tainted) 

fsv_error_handler(); 

} 

Here,  the  error  keyword  signals  to  the  com¬ 
piler  that  special  action  is  required  when 
the  condition  is  met. 

Unlike  most  taint  tracking  systems,  the 
security  policy  (lattice  and  summary  decla¬ 
rations)  and  underlying  analysis  is  not 
hard-coded  into  the  compiler  or  runtime 
system.  Instead,  this  semantic  information 
for  the  analysis  is  provided  in  a  separate 
specification  file.  This  separation  allows 
the  system  to  remain  general  and  flexible 
enough  to  enforce  a  wide  variety  of  secu¬ 
rity  policies. 

Some  insight  into  the  generality  and 
flexibility  of  the  DDFA  system  can  be 
demonstrated  by  how  it  guards  against  file 
disclosure  attacks  which  cannot  be  handled 
with  traditional  taint  tracking  systems.  For 
the  analysis,  two  lattices  must  be  defined 
as  follows  to  track  the  origin  of  data  and 
its  trustedness: 

property  Kind  :  {  File,  Filesystem,  Client, 
Server,  Pipe,  Command, 
StandardIO, 
Environment, 
Systeminfo,  NameServer 
} 

property  Trust :  { Remote,  External, 
Internal } 


For  a  true  file  disclosure  problem,  only 
Vile  is  used,  but  this  definition  could  be 
reused  for  other  policies  that  need  to  dis¬ 
tinguish  between  other  sources.  For  the 
sake  of  brevity,  summaries  for  how  lattice 
flow  values  are  introduced  into  the  system 
and  how  those  flow  values  are  propagated 
will  not  be  shown  here,  as  it  is  somewhat 
similar  to  the  previous  format  string  attack 
example.  The  following  specification 
defines  a  violation  of  the  policy  where  Vile 
data  from  a  file  with  Vemote  trustedness  is 
sent  to  a  server  socket  with  Vemote  trust 
(indicating  that  it  was  initiated  by  a  remote 
user): 

procedure  write(fd,  buf_ptr,  size) 

{ 

on  entry  { fd  ^  lOHandle,  buf_ptr  ^ 
buffer } 

error  if  ( (Trust :  buffer  could-be  Remote 
&&  Kind  :  buffer  could-be  File)  && 

(Trust :  lOHandle  could-be  Remote 
&&  Kind  :  lOHandle  could-be  Server) ) 
file_disclosure_error_handler() 

} 

Static  Data  Flow  Analysis 

To  avoid  the  cost  of  tracking  all  objects  at 
runtime,  the  DDFA  system  performs 
inter-procedural  data  flow  analysis  that 
identifies  all  program  locations  where  pol¬ 
icy  violations  might  occur.  A  subsequent 
inter-procedural  analysis  identifies  all  pro¬ 
gram  statements  that  affect  the  lattice  flow 
value  of  objects  that  may  trigger  a  policy 
violation.  Both  of  these  compiler  passes 
are  supported  by  a  fast  and  precise  point¬ 
er  analysis  for  potentially  aliased  data. 

The  first  pass  statically  identifies  all 
possible  violations  of  the  security  policy. 
If  the  analysis  can  prove  that  there  are  no 
such  vulnerabilities  in  the  program,  no 
further  analysis  or  instrumentation  is 
needed.  When  analysis  determines  that  a 
vulnerability  exists  or  cannot  determine  if 
a  vulnerability  is  genuine,  additional  analy¬ 
sis  is  needed  to  determine  where  instru¬ 
mentation  is  required. 

The  second  pass  is  an  inter-procedural 
dependence  analysis  that  identifies  all  the 
statements  in  the  program  that  affect  the 
flow  value  of  objects  that  may  trigger  a 
security  violation.  This  pass  provides  a  list 
of  statements  to  the  instrumentation 
engine  to  be  inserted  into  the  original 
source  code  so  that  dynamic  data  flow 
analysis  can  be  performed  at  runtime. 

Pointer  analysis  is  necessary  for  mem¬ 
ory-unsafe  languages  like  C  and  C++ 
because  objects  could  have  many  different 
pointers  pointing  to  them,  making  it  trick¬ 
ier  to  reason  precisely  about  the  flow  of 
data.  The  limited  scalability  of  pointer 


analysis  has  stymied  previous  attempts  to 
apply  inter-procedural  analysis  to  dynamic 
taint  tracking  [8].  However,  the  DDFA 
system  makes  use  of  a  highly  scalable  and 
accurate  client-driven  pointer  analysis  that 
leverages  the  semantic  information  pro¬ 
vided  by  the  security  policy  to  dramatical¬ 
ly  reduce  the  amount  of  code  that  needs 
to  be  instrumented  [9,  10]. 

Instrumentation  Engine 

The  instrumentation  engine  uses  the 
results  from  static  data  flow  analysis  to 
determine  where  in  the  program  addi¬ 
tional  code  must  be  inserted  to  support 
the  DDFA  at  runtime.  The  instrumenta¬ 
tion  engine  also  uses  the  semantics  of  the 
security  policy  specification  to  determine 
which  particular  calls  from  the  DDFA 
runtime  library  will  be  inserted  into  the 
program.  Continuing  our  format  string 
attack  example,  the  following  C  code 
snippet  shows  how  the  network  library 
call  recvfj  would  be  instrumented  if  static 
analysis  determines  that  it  may  introduce 
suspect  data  into  the  system: 

recv(sock_fd,  recv  msg,  10, 0); 
ddfaJnsert(DDFA_LTAINT,  recv  msg, 
strlen(recv_msg),  DDFA_ 
LTAINT_TAINTED); 

The  ddfa_insert()  call  sets  the  memory 
region  occupied  by  the  string  object 
recv_msg  as  TAINTED.  Any  attempt  to 
copy  the  data  occupied  by  this  object  to 
another  memory  region  or  to  use  this  data 
in  an  unsafe  manner  would  pass  the  flow 
value  tainted  to  the  copied  object  or  trig¬ 
ger  the  error  handler  respectively.  To  sup¬ 
port  propagation  of  tainted  data,  the  fol¬ 
lowing  code  snippet  shows  how  the  library 
call  StrdupQ  would  be  instrumented: 

copy_msg  =  strdup(recv_msg); 
ddfa_copy_flowval(DDFA_LTAINT, 
copy_msg,  recv  msg,  strlen(copy_msg)); 

Finally,  the  following  code  snippet  shows 
how  the  library  call  prinfQ  would  be 
instrumented  to  conditionally  invoke  an 
error  handler  before  unsafe  use  of  tainted 
data  occurs: 

if  ( ddfa_check_flowval(DDFA_LTAINT, 
copy_msg,  DDFA_LTAINT_TAINTED) ) 
{fsverrorhandlerO;} 
printf(copy_msg); 

This  example  illustrates  how  the 
DDFA  approach  can  go  much  further 
than  simply  detecting  vulnerabilities,  as 
with  purely  static  approaches,  by  allowing 
the  original  program  to  execute  securely 
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Program 

Original 

DDFA 

Runtime  Overhead 

pfingerd 

3.07  seconds  (s) 

3.19  s 

3.78  % 

muh 

1 1 .23  milliseconds  (ms) 

11.23  ms 

0.0  % 

wu-ftpd 

2.745  megabytes  per  second  (MB/s) 

2.742  MB/s 

<  1  % 

bind 

3.580  ms 

3.566  ms 

<  1% 

apache 

6.062  MB/s 

6.062  MB/s 

<  1% 

Average  Overhead:  0.65  % 

Table  2:  ^kuntime  Overhead  for  Server  Programs  Pefbrming  Taint  Analysis 


even  if  the  programmer  does  not  fix  the 
underlying  problem.  Additionally,  the  flow 
of  the  original  program  is  not  affected 
unless  a  potential  unsafe  use  of  tainted 
data  is  detected. 

DDFA 

The  DDFA  is  supported  by  the  data  flow 
analysis  at  runtime  library  and,  for  each 
type  of  analysis,  tracks  the  lattice  flow  val¬ 
ues  for  aU  objects  identified  by  the  static 
analysis  as  potentially  unsafe.  As  the  pro¬ 
gram  executes,  our  system  tracks  object 
flow  values  at  the  byte  level,  which  pro¬ 
vides  fine-grained  tracking  that  is  neces¬ 
sary  in  memory  unsafe  languages  such  as 
C.  If  an  unsafe  use  of  an  object  occurs, 
the  error  handler  specified  in  the  security 
policy  will  be  invoked  before  the  object  is 
actually  used  in  an  unsafe  manner.  The 
primary  benefit  of  performing  data  flow 
analysis  at  runtime  is  to  subvert  a  security 
attack  by  invoking  the  appropriate  error 
handler  specified  in  the  security  policy 
before  an  unsafe  use  of  an  object  occurs. 

Effectiveness  of  the  DDFA 
Approach 

In  this  section  we  wiU  address,  in  both 
objective  and  subjective  terms,  the  effec¬ 
tiveness  of  the  DDFA  approach. 

Minimizing  Impact  on  Development 
Processes 

One  of  the  most  important  benefits  in 
using  the  DDFA  system  is  that  it  works 
on  existing  C  programs  and  does  not 
require  the  developer  to  make  any 
changes  to  the  original  source  code.  The 
developer  need  only  recompile  their  pro¬ 
gram  with  the  DDFA  system  to  create  a 
security-enhanced  version  of  their  origi¬ 
nal  source  code.  If  not  using  one  of  the 
default  policy  specifications  provided  by 
our  system,  such  as  taint  tracking  or  file  dis¬ 
closure  vulnerability,  a  security  expert  can 
extend  the  system  by  creating  a  new  pol¬ 
icy  specification  file.  In  contrast,  to 
extend  a  conventional  taint  tracking  sys¬ 
tem  to  something  other  than  taint  track¬ 
ing,  core  components  of  the  compiler 
infrastructure  would  have  to  be  rewrit¬ 
ten.  In  the  DDFA  system,  the  only 
change  is  in  the  policy  specification  file 
itself.  This  also  allows  new  security  poli¬ 
cies  to  be  developed  quickly  in  response 
to  new  attacks,  resulting  in  a  more  agile 
response  to  the  ever-changing  security 
environment. 

Although  the  DDFA  approach  cur¬ 
rently  requires  the  source  code  of  the 
application  as  input  to  the  system  instead 
of  the  binary  executable,  it  does  not 


require  the  implementation  source  code 
of  the  library  functions  and  system  calls 
that  appear  as  summaries  in  the  policy 
specification  file.  Only  an  understanding 
of  the  behavior  of  the  function  calls  is 
required.  Moreover,  the  work  described 
in  this  article  is  part  of  an  ongoing 
research  program,  which  has  as  a  longer- 
term  goal  of  applying  the  DDFA 
approach  to  binary  executables.  This 

'^One  of  the  most 
important  benefits  in 
using  the  DDFA  system 
is  that  it  works  on 
existing  C  programs  and 
does  not  require  the 
developer  to  make  any 
changes  to  the 
original  source  code/* 

could  be  an  evolutionary  step  in  the 
research,  since  the  DDFA  system  already 
performs  its  analysis  on  a  low-level  repre¬ 
sentation  of  the  input  program. 
However,  there  are  challenges  in  moving 
to  binary  because  significant  information 
that  is  leveraged  in  minimizing  the  per¬ 
formance  impacts  of  security  insertion  is 
lost  in  the  compilation  process. 

Another  related  aspect  of  our  system 
is  that  the  security  enforcement  is  added 
to  the  program  after  it  has  been  devel¬ 
oped.  This  opens  up  many  new  software 
engineering  possibilities  such  as  code 
separation.  In-house  software  will  be 
more  maintainable  and  agile  because  it  is 
unfettered  by  security  concerns.  Likewise, 
commercial  off-the-shelf  and  open 
source  software  can  be  brought  into  a 
highly  trusted  environment  and  then 
automatically  made  more  secure.  It  also 
allows  an  organization  to  keep  their  secu¬ 


rity  policy  specification  private  when  sub¬ 
contracting  software  development  to  out¬ 
side  organizations. 

Minimizing  Performance  Overhead 

Another  important  benefit  is  that  the 
DDFA  approach  takes  advantage  of  the 
fact  that  only  a  very  small  portion  of  a 
program  is  actually  involved  in  any  given 
security  attack  [11].  Identifying  this  small 
portion  of  the  program,  however,  requires 
sophisticated  static  analysis.  Without  this 
analysis,  large  portions  of  the  program 
would  have  to  be  instrumented,  substan¬ 
tially  increasing  the  program’s  runtime 
overhead. 

In  order  to  quantify  the  runtime 
overhead  that  is  incurred  on  programs 
enhanced  by  the  DDFA  system,  we  mea¬ 
sured  the  runtime  overhead  for  two  dif¬ 
ferent  sets  of  open-source  C  programs.  In 
the  first  set,  we  measured  response  time 
or  throughput  for  several  different  serv¬ 
er  type  programs  with  considerable 
input/output.  We  took  measurements 
against  the  original  program  and  then 
again  after  instrumentation  by  the 
DDFA  system  (in  this  case,  applying  a 
taint-based  policy  specification).  As 
shown  in  Table  2,  our  solution  has  an 
average  overhead  of  0.65  percent.  The 
current  fastest  compiler-based  and 
dynamically  optimized  systems  report 
server  application  overhead  of  3-7  per¬ 
cent,  and  6  percent,  respectively  [12,  13]. 

In  the  second  test  set,  we  measured 
Standard  Performance  Evaluation 
Corporation  (SPEC)  workloads  of  four 
SPECint  2000  benchmarks  that  are 
compute-bound  applications  after  per¬ 
forming  a  format  string  vulnerability 
analysis.  In  each  case,  our  static  analysis 
determines  that  the  programs  contain 
no  such  vulnerability,  as  expected.  Thus, 
the  true  overhead  for  these  examples  is 
zero  percent.  In  order  to  understand  the 
impact  of  our  system  on  compute- 
bound  programs  that  do  contain  vulner¬ 
abilities,  we  manually  inserted  a  vulnera¬ 
bility  into  each  of  the  benchmarks.  As 
can  be  seen  in  Table  3  (see  next  page), 
the  average  runtime  overhead  is  less 
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Program 

Runtime  Overhead 

gzip 

51 .35  % 

vpr 

<  1  % 

mcf 

<  1% 

crafty 

<  1% 

Average: 

12.9% 

Table  3:  ^kuntime  Overhead for  Compute- 
Hound  Programs  Peiforming  Format  String 
Vulnerability 


than  13  percent,  which  is  significantly 
better  than  the  best  previously  reported 
averages  of  75-260  percent  [12,  13], 

Minimizing  Code  Expansion 

Because  our  system  adds  instrumentation 
to  the  source  program,  it  expands  the  pro¬ 
grams  static  code  size.  To  quantify  this 
property,  we  measured  code  expansion  by 
comparing  the  sizes  of  the  original  and 
modified  binary  executables  after  per¬ 
forming  taint  tracking  on  the  server  pro¬ 
grams  mentioned  previously.  As  can  be 
seen  in  Table  4,  the  average  expansion  is 
less  than  1  percent.  Compiler-based  sys¬ 
tems  such  as  the  GNU  Image-Finding 
Tool  report  30-60  percent  increases  in 
binary  size  [8]. 

Related  Capabilities  and  Future 
Directions 

Finally,  another  important  benefit  of  the 
DDFA  system  is  that  the  technology  is 
applicable  to  future  threats  and  other 
areas  that  are  not  specific  to  security.  For 
example,  systems  that  depend  on  seman¬ 
tic  information  such  as  information  flow 
(i.e.,  privacy  concerns)  or  access  controls 
can  be  enhanced  by  our  system  without 
modifying  the  core  infrastructure.  We 
recendy  demonstrated  this  generality  by 
showing  that  the  policy  specification  file 
can  be  used  to  define  roles  in  a  RBAC 
system,  and  then  subsequendy  applied  to 
a  set  of  software  that  previously  did  not 
have  RBAC.  The  DDFA  system  can  go 
beyond  problems  such  as  buffer  over¬ 


flows  and  overwrite  attacks  that  condnue 
to  plague  legacy  languages  and  solve 
problems  affecdng  even  safer  languages 
such  as  Java  and  C#.  These  languages  are 
not  immune  to  attacks  like  SQL  injection 
and  cross-site  scripting  that  depend  on 
semandc,  not  language-level,  errors  in 
data  handling.  DDFA,  along  with  the 
semandc  informadon  captured  by  the 
policy  specificadon,  can  address  these 
pqjes  of  problems. 

Our  challenge,  therefore,  is  to  devel¬ 
op  soludons  that  can  both  be  applied  to 
exisdng  legacy  software  today  for  imme¬ 
diate  benefits  while  also  looking  forward 
to  the  more  sophisticated  challenges  that 
face  us  in  the  future.  We  believe  that  the 
DDFA  system  is  well  positioned  to  pro¬ 
vide  a  practical  approach  to  enhancing 
the  security  of  legacy  software  in  the  near 
future.  We  are  also  continuing  our 
research  in  increasing  the  scalability  of 
pointer  analysis,  integrating  language- 
independence  into  the  DDFA  technolo¬ 
gy,  as  well  as  researching  and  testing  the 
breadth  of  applicability  of  the  approach 
its  elf.  ♦ 
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Table  4:  Static  Code  Expansion  After  Peiforming  Faint  Fracking 


Program 

Originai 

(bytes) 

DDFA 

(bytes) 

Code  Overhead 

pfingerd 

49,655 

49,655 

0.0  % 

muh 

59,880 

60,488 

1.01  % 

wu-ftpd 

205,487 

207,997 

1.22% 

bind 

215,669 

219,765 

1.90% 

apache 

552,114 

554,514 

0.43  % 

Average  Overhead: 

0.91  % 
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Web  Sites 


Department  of  Homeland  Security’s 
(DHS)  Software  Assurance  Program 

www.us-cert.gov/swa 

The  DHS  Software  Assurance  Program  spearheads  the  develop¬ 
ment  of  practical  guidance  and  tools  and  promotes  research  and 
development  of  secure  software  engineering,  examining  a  range 
of  development  issues  from  new  methods  that  avoid  basic  pro¬ 
gramming  errors  to  enterprise  systems  that  remain  secure  when 
portions  of  the  system  software  are  compromised.  Through  col¬ 
laborative  software  assurance  efforts,  stakeholders  seek  to  reduce 
software  vulnerabilities,  minimize  exploitation,  and  address 
ways  to  improve  the  routine  development  and  deployment  of 
trustworthy  software  products. 

The  Open  Web  Application  Security 
Project  (OWASP) 


www.owasE 


The  OWASP  is  a  worldwide  free  and  open  community  focused 
on  improving  the  security  of  application  software.  Their  mis¬ 
sion  is  to  make  application  security  “visible”  so  that  people  and 
organizations  can  make  informed  decisions  about  application 
security  risks. 

SecurityFocus 

www.securityfocus.com 

SecurityFocus  provides  the  most  comprehensive  and  trusted 
source  of  computer  and  application  security  information  on  the 
Internet.  It  provides  objective,  timely,  and  comprehensive  secu¬ 


rity  information  to  all  members  of  the  security  community 
Information  includes  computer  security  vulnerability 
announcements,  security-related  news  stories  and  feature  arti¬ 
cles,  effective  security  measure  implementation  ideas,  and  a  high 
volume,  full-disclosure  mailing  list  for  detailed  discussions. 

MILS:  High-Assurance  Security  at 
Affordabfe  Costs 

www.cotsiournalonline.com/home/article.php?id=  100423&pg=  1 
This  COTS  Journal  article  explores  how  multiple  independent 
levels  of  security  (MILS)  build  high-assurance  systems  that  must 
survive  high-threat  environments.  The  central  idea  behind 
MILS  is  to  partition  a  system  in  such  a  way  that  1)  the  failure 
or  corruption  of  any  single  partition  cannot  affect  any  other 
part  of  the  system  or  network,  and  2)  each  partition  can  be  secu¬ 
rity-evaluated  and  certified  separately. 

Build  Security  In  (BSI) 

http://buildsecurityin.us-cert.gov 

BSI  contains  and  links  to  best  practices,  tools,  guidelines,  rules, 
principles,  and  other  resources  that  software  developers,  archi¬ 
tects,  and  security  practitioners  can  use  to  build  security  into 
software  in  every  phase  of  its  development.  BSI  content  is  based 
on  the  principle  that  software  security  is  fundamentally  a  soft¬ 
ware  engineering  problem  and  must  be  addressed  in  a  systemat¬ 
ic  way  throughout  the  software  development  life  cycle. 
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Building  Secure  Systems  Using  Model-Based 
Engineering  and  Architectural  Models 

Dr.  Jorgen  Hansson,  Dr.  Peter  H.  Feiler,  and  John  Morley 

Software  Engineering  Institute 

The  Department  of  Defense’s  policy  of  multi-level  security  (MES)  has  long  employed  the  T>ell-EaPadula  and  Tiba  approach¬ 
es for  confidentiality  and  integrity;  more  recently,  the  multiple  independent  levels  of  security  j  safely  (MILS)  approach  has  been 
proposed.  These  approaches  allow  designers  of  software-intensive  systems  to  specify  security  levels  and  requirements  for  access 
to  protected  data,  but  thy  do  not  enable  them  to  predict  runtime  behavior.  In  this  article,  model-based  engineering  (MBE) 
and  architectural  modeling  are  shown  to  be  a  plaform  for  multi-dimensional,  multi-fidelity  analysis  that  is  conducive  for  use 
with  Bell-EaPadula,  Biba,  and  MIES  approaches,  and  enables  a  system  designer  to  exercise  various  architectural  design 
options  for  confidentiality  and  data  integrity  prior  to  system  realisation.  In  that  way,  MBE  and  architectural  modeling  can 
be  efficiently  used  to  validate  the  security  of  system  architectures  and,  thus,  gain  confidence  in  the  system  design. 


System  designers  face  several  challenges 
when  specifying  security  for  distrib¬ 
uted  computing  environments  or  migrat¬ 
ing  systems  to  a  new  execution  platform. 
Business  stakeholders  impose  constraints 
due  to  cost,  time-to-market  requirements, 
productivity  impact,  customer  satisfaction 
concerns,  and  so  forth.  Thus,  a  system 
designer  needs  to  understand  require¬ 
ments  regarding  the  confidentiality  and 
integrity  of  protected  resources  (e.g., 
data).  Additionally,  a  designer  needs  to 
predict  the  effect  that  security  measures 
win  have  on  other  runtime  quality  attrib¬ 
utes  such  as  resource  consumption,  avail¬ 
ability,  and  real-time  performance.  After 
all,  the  resource  costs  associated  with 
security  can  easily  overload  a  system. 
Nevertheless,  security  is  often  studied  only 
in  isolation  and  late  in  the  process. 
Furthermore,  the  unanticipated  effects  of 
design  approaches  or  changes  are  discov¬ 


ered  only  late  in  the  life  cycle  when  they 
are  much  more  expensive  to  resolve’. 

MBE  for  Security  Analysis 

Modeling  of  system  quality  attributes, 
including  security,  is  often  done  —  when  it 
is  done  -  with  low-fidelity  software  mod¬ 
els  and  disjointed  architectural  specifica¬ 
tions  by  various  engineers  using  their  own 
specialized  notations.  These  models  are 
typically  not  maintained  or  documented 
throughout  the  life  cycle,  making  it  diffi¬ 
cult  to  obtain  a  system  view.  Flowever,  a 
single-source  architecture  model  of  the 
system  that  is  annotated  with  analysis-spe¬ 
cific  information  allows  changes  to  the 
architecture  to  be  reflected  in  the  various 
analysis  models  with  Uttie  effort;  those 
models  can  easily  be  regenerated  from  the 
architecture  model  (Figure  1).  This 
approach  also  allows  the  designer  to  con¬ 
duct  an  adequate  trade-off  analysis  and 


evaluate  architectural  variations  prior  to 
system  realization,  thereby  gaining  confi¬ 
dence  in  the  architectural  design.  Models 
also  can  be  used  to  evaluate  the  effects  of 
reconfiguration  and  system  revisions  in 
post-development  phases. 

Using  MBE  tools,  the  Software 
Engineering  Institute  (SEI)  has  developed 
analytical  techniques  to: 

•  Represent  standard  security  protocols 
for  enforcing  confidentiality  and 
integrity,  such  as  BeU-LaPadula  [1,  2], 
Chinese  wall  [3,  4],  role-based  access 
control  [5],  and  the  Biba  model  [6]. 

•  Model  and  validate  security  using  sys¬ 
tem  architecture  according  to  flow- 
based  approaches  early  and  often  in  the 
life  cycle. 

The  MBE  tools  that  the  SEI  uses  are 
the  Architecture  Analysis  and  Design 
Language  (AADL)  and  the  Open  Source 
AADL  Tool  Environment  (OSATE)  set  of 
analysis  plugins  [7]”.  The  AADL  is  used  to 
model  and  document  system  architecture 
and  provide  the  following  platform  for 
analyses: 

•  Using  a  single  architecture  model  to 
evaluate  multiple  quality  attributes, 
including  security. 

•  Early  and  often  during  system  design  or 
when  upgrading  existing  system  archi¬ 
tecture. 

•  At  different  architecture  refinement  lev¬ 
els  as  information  becomes  available. 

•  Along  diverse  architectural  aspects, 
such  as  behavior  and  throughput. 

Architectural  Considerations 

Security  as  an  architectural  concern 
crosscuts  all  levels  of  the  system  (appli¬ 
cation,  middleware,  operating  systems, 
and  hardware).  Thus,  security  requires 
intra-  and  inter-level  validation  and  has 
immediate  effects  on  the  runtime  behav¬ 
ior  of  the  system,  specifically  on  other 
dependability  attributes. 


Figure  1:  A  Single,  System  Architectural  Model  Annotated  for  Multple  Mon-Eunctional  Quality 
Analyses 
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The  designer  needs  to  enforce  intra- 
and  inter-level  security  throughout  the 
architecture.  Figure  2  depicts  various  sys¬ 
tem  levels  involved  in  the  validation  of 
security  privileges  against  confidentiality 
requirements  (it  assumes  that  authentica¬ 
tion  and  other  necessary  security  services 
are  enforced).  The  designer  seeks  to  ensure 
that  the  software  applications  do  not  com¬ 
promise  the  confidentiality  of  the  secure 
information  they  are  exchanging.  Conse¬ 
quentially,  software  applications  need  to 
execute  on  top  of  a  secure  operating  sys¬ 
tem,  be  mapped  to  a  protected  and  secured 
hardware  memory  space,  and  communicate 
over  a  secure  communication  channel.  If 
the  data  is  labeled  “confidential,”  then 
every  architectural  layer  needs  to  have  a 
clearance  of  at  least  that  level. 

Additionally,  the  designer  needs  to 
acknowledge  that  security  comes  with  a 
cost.  Encryption,  authentication,  security, 
and  protection  mechanisms  increase  band¬ 
width  demand  in  terms  of  the  central  pro¬ 
cessing  unit  (CPU),  the  network,  and  mem¬ 
ory.  These  increases  affect  the  temporal 
behavior  of  the  system  (worst-case  execu¬ 
tion  time,  response  time,  schedulabiUty,  and 
end-to-end  latency)  as  well  as  power  con¬ 
sumption  (especially  important  in  battery- 
driven  or  limited  lifetime  devices  such  as 
sensor  networks  or  portable  communica¬ 
tion  devices). 

As  a  result,  security  cannot  be  consid¬ 
ered  in  isolation.  The  system  designer 
makes  choices  to  trade  these  quality  attrib¬ 
utes  against  each  other  (a  particular  concern 
for  embedded  and  real-time  systems,  which 
operate  under  significant  resource  con¬ 
straints  while  ensuring  high  levels  of 
dependability  and  security).  Security  is  inter¬ 
linked  with  other  non-functional  behaviors 
such  as  predictability/ timeliness  and 
resource  consumption,  as  well  as  inadver¬ 
tent  effects  on  reliability  and  availability. 
Figure  3  illustrates  some  of  those  depen¬ 
dencies  on  the  single-model,  multiple-analy¬ 
sis  view. 

An  MBE  Approach  to 
Validating  Confidentiality 

Confidentiality  addresses  concerns  that 
sensitive  data  should  be  disclosed  to  or 
accessed  only  by  authorized  users  (i.e., 
enforcing  prevention  of  unauthorized  dis¬ 
closure  of  information).  Data  integrity  is 
closely  related,  as  it  concerns  prevention  of 
unauthorized  modifications  of  data. 

To  model  and  validate  the  confidential¬ 
ity  of  a  system,  we  distinguish  between 
general  and  application-dependent  valida¬ 
tion.  General  validation  of  confidentiality  is 
the  process  of  ensuring  that  a  modeled  sys- 
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Figure  2:  System  Perspectwe  on  Security 

tern  conforms  to  a  set  of  common  condi¬ 
tions  that  support  system  confidentiality 
independent  of  a  specific  reasoning  frame¬ 
work  for  security.  MBE  takes  advantage  of 
the  versatile  concept  of  subjects  operating 
on  objects  by  permissible  access  (read,  exe¬ 
cute,  append,  and  write),  a  notion  intro¬ 
duced  by  Bell  and  LaPadula  [1],  enabling  us 
to  model  and  validate  security  at  both  the 
software  and  hardware  levels. 

This  form  of  validation  assumes  that 
subjects  and  objects  are  assigned  a  security 
level  that  is  the  minimum  representation  to 
enforce  basic  confidentiality  and  need-to- 
know  principles.  By  contrast,  application- 
specific  validation  relies  on  detailed  confi¬ 
dentiality  requirements  and  a  specific,  rea¬ 
soning-based  security  framework. 


The  MBE  security  framework  features: 

•  Representation  of  the  confidentiality 

requirements  of  resources  (i.e.,  objects). 

•  Representation  and  generation  of  secu¬ 
rity  clearance/least  privileges’  of  sub¬ 
jects  operating  on  the  objects. 

•  Representation  of  authorized  opera¬ 
tions,  ensuring  unauthorized  infiltra¬ 

tion,  unauthorized  exfiltration,  and 
unauthorized  median  of  actions.  This  is 
captured  in  an  access  matrix. 

With  the  object’s  security  requirements 
specified  in  an  AADL  model,  the  least 
amount  of  privileges  for  the  subjects  can 
be  generated  in  a  straightforward  manner. 
Given  that  the  subjects’  privileges  are 
specified,  a  mismatch  between  the  least 
privilege  and  what  has  been  specified 


Figure  3:  Singk  Architectural  Model  Showing  an  Example  of  Impact  on  and  Interaction  of  Non- 
Functional  Behavior  Due  to  a  Change  in  Security 
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—  Property  intended  to  be  customized  by  modelers. 

—  Parameterizes  the  security  property  definitions, 
property  set  Security_Types  is 

—  Military  levels  by  default 
Classifications : 

Type  enumeration  (unclassified,  confidential,  secret, 

top_secret ) ; 

—  This  must  be  the  first  element  of  Classifications 
Def ault_Classif ication : 

constant  Security_Types : :Classifications  => 

—  Default  set  of  categories 
Categories : 

type  enumeration  (A,  B,  C,  D); 
end  Security_Types ; 

Figure  4:  Specification  of  Security  'Levels 


means  the  assigned  privilege  is  either 
insufficient  or  greater  than  the  minimum 
privilege.  The  latter  result  may  be  unnec¬ 
essary  or  an  indication  that  the  subject 
might  be  associated  with  objects  not  yet 
described  in  the  model. 

The  following  types  of  security  valida¬ 
tion  and  analysis  are  available  as  OSATE 
plugins: 

•  Basic  confidentiality  principle.  Ac¬ 
cess  should  only  be  granted  if  given  the 
appropriate  security  clearance. 

•  Need-to-know  principle.  Access 
should  be  granted  to  a  resource  only  if 
there  is  a  need. 

•  Controlled  sanitization.  Lowering  the 
security  level  of  an  object  or  subject 
should  only  be  authorized  and  per¬ 
formed  by  a  privileged  subject. 

•  Non-alteration  of  object’s  security 
requirements.  A  subject  using  an 
object  as  input  should  not  alter  the  secu¬ 
rity  level  of  the  object,  even  if  the  object 
is  updated  as  an  output  from  the  subject. 

•  Hierarchical  conditions.  A  compo¬ 
nent  has  (1)  a  security  level  that  is  the 
maximum  of  the  security  levels  of  its 


subcomponents,  and  (2)  aU  connections 
are  checked  to  determine  whether  the 
source  component  of  a  connection 
declaration  has  a  security  level  that  is 
the  same  or  lower  than  that  of  the  des¬ 
tination  component. 

Using  OSATE  and  the  AADL,  system 
designers  and  developers  can  add  analysis 
techniques  as  needed. 

The  validation  through  architectural 
modeling  of  system  security  —  given  the 
confidentiality  requirements  of  data  objects 
and  the  security  clearance  by  users  —  must 
include  validation  of  (1)  software  architec¬ 
ture,  and  (2)  system  architecture  where  the 
software  architecture  is  mapped  to  hard¬ 
ware  components. 

By  mapping  the  entities  of  a  software 
architecture  (e.g,  processes,  threads,  and 
partitions)  to  a  hardware  architecture  (con¬ 
sisting  of,  for  example,  CPUs,  communica¬ 
tion  channels,  and  memory),  we  can  ensure 
that  the  hardware  architecture  supports 
required  security  levels,  as  described  in 
Figures  4  and  5. 

Consider  the  scenario  of  two  commu¬ 
nicating  processes,  both  requiring  a  high 


level  of  security  because  the  data  objects 
require  secret  clearance.  The  system  plat¬ 
form  in  this  scenario  consists  of  a  set  of 
CPUs  with  hardware  support  for  various 
algorithms  that  encrypt  messages  before 
network  transmission.  By  modeling  the  sys¬ 
tem,  we  can  represent  and  validate  that 
both  processes  and  threads  (now  consid¬ 
ered  to  be  objects)  can  be  executed  (access 
mode)  on  CPUs  (subjects)  with  adequate 
encryption  support.  Furthermore,  we  can 
validate  that  CPUs  (objects)  communicate 
data  (access  modes  of  writing  and  reading) 
over  appropriately  secured  communication 
channels  (subjects).  In  a  similar  fashion,  we 
can  enforce  design  philosophies  saying  that 
only  processes  of  the  same  security  level 
are  allowed  to  co-exist  within  the  same 
CPU  or  partition,  or  that  they  can  write  to 
a  secured  memory. 

A  combination  of  the  AADL  and  the 
OSATE  security  plugin  tool  has  been  put 
into  use  in  industry.  RockweU-CoUins  used 
the  technology  to  enable  the  high-assur- 
ance  handling  of  data  from  multiple  sen¬ 
sors  having  varying  levels  of  security,  such 
as  airborne  imagery  with  the  Field 
Programmable  Gate  Array  (FPGA). 
Typically,  a  high-assurance  processor  is 
used  to  securely  tag  variable  input.  An 
FPGA  is  powerful  and  fast.  It  is  deemed 
easier  to  develop  applications  on  an  FPGA, 
which  also  reduces  the  cost  and  time-to- 
market.  Furthermore,  the  FPGA  can  be 
reprogrammed  at  runtime  (e.g.,  to  fix  bugs), 
which  can  lower  maintenance-engineering 
costs.  Because  FPGA  behavior  is  more 
complex,  architecture-level  definition  and 
analysis  are  needed.  To  this  end,  RockweU- 
CoUins  developed  architectural  models  of 
the  FPGA  using  AADL  and  used  the 
OSATE  tool  to  validate  security  and 
demonstrate  the  high-assurance  potential 
of  FPGAs. 

Validating  MILS  Architectures 
With  the  MBE  Approach 

The  AADL  and  OSATE  tools  can  be  used 
to  validate  the  security  of  systems  designed 
using  the  MILS'*  architecture  approach  (see 
[8,  9]).  MILS  uses  two  mechanisms  to  mod¬ 
ularize  —  or  divide  and  conquer  —  in  archi¬ 
tecting  secure  systems:  partitions,  and  sepa¬ 
ration  into  layers.  The  MILS  architecture 
isolates  processes  in  partitions  that  define  a 
collection  of  data  objects,  code,  and  system 
resources  and  can  be  evaluated  separately. 
Each  partition  is  divided  into  the  following 
three  layers,  each  of  which  is  responsible  for 
its  own  security  domain  and  nothing  else: 

1.  Separation  Kernel  (SK).  Responsible 
for  enforcing  data  isolation,  control  of 
information  flow,  periods  processing. 


Figure  5:  Architectural  Components  to  'Which  Security  'Levels  and  Requirements  Can  Be  Connected 

property  set  Security_Attributes  is 

Class:  inherit  Security_Types : rClassifications  => 
value ( Security_Types : : Def ault_Classif ication ) 
applies  to  (data,  subprogram,  thread,  thread  group, 
process,  memory,  processor,  bus,  device, 
system,  port,  server  subprogram, 
parameter,  port  group); 

Category:  inherit  list  of  Security_Types :: Categories  => 
() 

applies  to  (data,  subprogram,  thread,  thread  group, 
process,  memory,  processor,  bus,  device, 
system,  port,  server  subprogram, 
parameter,  port  group); 

end  Security_Attributes ; 
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and  damage  limitation.  An  example  is 
the  SK  Protection  Profile  [8], 

2.  Middleware  service  layer. 

3.  Application  layer. 

Thus,  the  MILS  separates  security 
mechanisms  and  concerns  into  the  follow¬ 
ing  three  component  types,  classified  by 
how  they  process  data: 

•  Single-Level  Secure  (SLS).  Processes 
data  at  one  security  level. 

•  Multiple  Single-Level  Secure  (MSLS). 
Processes  data  at  multiple  levels,  but 
maintains  separations  between  classes 
of  data. 

•  MLS.  Processes  data  at  multiple  levels 
simultaneously  and  transforms  data 
from  one  level  to  another. 

The  strength  of  the  MILS  architecture 
lies  in  its  reductionist  approach  to  decom¬ 
pose  a  system  into  components  of  the 
above-mentioned  types  that  would  be  more 
manageable  to  certify.  These  components 
are  also  mapped  to  partitions  (and,  as  men¬ 
tioned  earlier,  the  MILS  architecture 
approach  builds  on  partitioning  as  one  key 
concept  to  enforce  damage  limitation  and 
separation  of  time  and  space) . 

An  MBE  approach  is  conducive  to  the 
validation  concerns  most  critical  to  MILS, 
including: 

•  Validating  the  structural  rigidity  of 
architecture,  such  as  the  enforce¬ 
ment  of  legal  architectural  refine¬ 
ment  patterns  of  a  security  compo¬ 
nent  into  SLS,  MSLS,  and  MLS 
types.  Given  that  an  MILS  architecture 
design  and  system  is  decomposed  into 
security  components  that  can  be  certi¬ 
fied  in  isolation,  the  strucmral  rigidity 
concerns  the  legal  mappings  and  con¬ 
nections  of  the  components.  The 
decomposition  into  SLS,  MSLS,  and 
MLS  types  can  be  applied  to  compo¬ 
nents,  connectors,  and  ports. 
Furthermore,  each  component  can  be 
divided  into  parts  using  the  product, 
cascade,  or  feedback  decomposition 
patterns  [10,  11,  12].  For  example,  an 
MSLS  component  with  n  security  levels 
can  be  decomposed  into  n  distinct  SLS 
components.  Thus,  confidence  in  the 
validation  of  an  architecmre  increases 
with  the  fidelity  of  the  modeling.  By 
using  an  architectural  model  in  AADL 
to  capture  the  security  types  and  multi¬ 
ple  architecmral  levels,  MBE  analysis  is 
conducted  to  validate  the  correctness  of 
the  decompositions  and  mappings. 

•  Architectural  modeling  and  valida¬ 
tion  of  assumptions  underlying 
MILS.  Fundamental  to  enforcement  of 
security  in  an  MILS  architecture  is  hav¬ 
ing  a  system  that  supports  partitioning, 
specifically  damage  limitation  and  sepa¬ 


ration  in  time  and  space.  By  partitioning 
the  system,  one  minimizes  the  risk  of 
illegal  component  interactions  among 
components  and  protects  components 
from  the  faulty  behavior.  This  can  be 
realized  in  the  system  architecmre  by 
ensuring  fault-containment  and  deploy¬ 
ing  security-cognizant  memory  alloca¬ 
tion  so  that  MILS  components  and 
tasks  reside  in  protected  memory 
spaces  -  and  do  not  co-reside  in  the 
same  memory  space  if  they  differ  in 
security  levels.  Similarly,  separation  in 
time  can  be  ensured  through  avoiding 
the  interleaved  execution  of  tasks  with 
different  security  levels,  realized  in  par¬ 
tition  scheduling  and  validating  execu¬ 
tion  behaviors.  The  AADL  supports 
the  modeling  of  partitions  and  virmal 
processors.  As  well,  the  virtual  machine 
mechanism  is  recognized  as  a  key  con¬ 
cept  for  providing  robusmess  through 
fault  containment  because  it  provides 
time  and  space  partitioning  to  isolate 
application  components  and  subsys¬ 
tems  from  affecting  each  other  (due  to 
sharing  of  resources).  This  architecmre 
pattern  can  be  found  in  the 
Aeronautical  Radio  Incorporated 
(ARINC)  653  standard  [13].  A  single¬ 
source  architectural  model  in  AADL 
can  thus  be  used  to  validate  the  security 
requirement  in  an  architectural  context, 
specifically  the  MILS  composition,  and 
the  architecmral  assumptions  required. 

•  Validating  requirements  specific  to 
the  NEAT  characteristics  and  the 
communication  system.  MILS  re¬ 
quires  that  its  SK  and  the  trusted  com¬ 
ponents  of  middleware  services  are 
implemented  so  that  the  security  capa¬ 
bilities  enforce  what  is  commonly 
referred  to  as  the  NEAT  characteristics: 
0  N  on-hypassahle.  Security  func¬ 
tions  cannot  be  circumvented. 

°  '^valuatahle.  The  size  and  com¬ 
plexity  of  the  security  functions 
allow  them  to  be  verified  and 
evaluated. 

°  Klwajs  invoked.  Security  func¬ 
tions  are  invoked  each  and 
every  time  without  exception. 

°  "Xamperproof.  Subversive  code 
cannot  alter  the  function  of  the 
security  functions  by  exhaust¬ 
ing  resources,  overrunning 
buffers,  or  other  forms  of  mak¬ 
ing  the  security  software  fail. 

The  MBE  approach  allows  designers  to 
assure  that  software  applications  execute  on 
top  of  a  secure  operating  system,  map  to  a 
protected  and  secured  hardware  memory 
space,  and  communicate  over  secure  com¬ 
munication  channels.  It  also  enables  the 


analysis  of  security  measures  early  and 
throughout  the  development  life  cycle. 

Conclusions 

The  objective  of  a  secure  system  implies 
that  security  clearances  are  given  conserva¬ 
tively.  The  MBE  approach  supports  this 
objective  through  enabling  analysis  of  the 
architectural  model  to  derive  the  minimum 
security  clearance  on  components.  By  pro¬ 
viding  mechanisms  to  ensure  that  sanitiza¬ 
tion  is  conducted  within  allowed  bound¬ 
aries,  the  MBE  approach  enables  the  sys¬ 
tem  designer  to  analyze  and  trace  more 
threatening  security  risks,  since  sanitizing 
actions  are  permitted  exemptions  of  securi¬ 
ty  criteria  and  rules,  and  as  such  should  be 
minimized  in  the  system. 

Security  analysis  using  the  MBE  ap¬ 
proach  also  supports: 

•  The  evaluation  of  an  architecmre  con¬ 
figuration  with  respect  to  impact  on 
other  non- functional  attributes,  such  as 
increases  in  power  consumption,  band¬ 
width  usage,  and  performance. 

•  The  validation  of  architecmral  require¬ 
ments  necessary  to  enforce  the  MILS 
approach  to  containing  faults,  through 
partitioning  and  separation  in  time  and 
space. 

•  A  reduction  of  the  effort  necessary  for 
re-certification  in  the  event  of  architec¬ 
tural  changes. 

Furthermore,  validation  of  security  can 
be  conducted  at  multiple  layers  and  differ¬ 
ent  levels  of  fidelity,  early  and  throughout 
the  development  life  cycle.^ 
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Notes 

1.  A  NIST  study  observed  that  70  percent 
of  aU  defects  are  introduced  prior  to 
implementation.  Yet  only  3.5  percent  of 
the  defects  were  detected  in  these  phas¬ 
es,  while  50.5  percent  of  the  faults  were 
detected  in  the  integration  phase.  The 
defect  removal  cost  ranged  from  5  to  30 
times  relative  to  the  cost  of  removing 
the  defect  in  the  phase  of  introduction 
(if  it  had  been  detected).  Other  sources 
are  reporting  similar  estimates;  while 
the  numbers  vary,  the  conclusions  do 
not  [14]. 

2.  The  AADL,  an  international  industry 
standard,  incorporates  an  XML/XMI 
exchange  format  to  support  model 
interchange  and  tool  chaining.  AADL 


also  can  be  used  (1)  with  UML  state  and 
process  charts  through  its  UML  profile, 
(2)  to  drill  into  root  causes  and  develop 
quantitative  analysis  as  a  follow-up  to 
the  SEI  Architecture  Tradeoff  Analysis 
Method®,  and  (3)  in  conjunction  with 
assurance  cases,  to  support  claims  made 
about  the  safety,  security,  or  reliability  of 
a  system.  The  freely  available  OSATE 
includes  analysis  plugins  for  perfor¬ 
mance,  resource  consumption,  security, 
and  reliability. 

3.  The  principle  of  least  privilege  has  been 
identified  as  important  for  meeting 
integrity  objectives;  it  requires  that  a 
user  (subject)  be  given  no  more  privi¬ 
lege  than  necessary  to  perform  a  job. 
This  principle  includes  identifying  what 
the  subject’s  job  requires  and  restricting 
the  subject’s  ability  by  granting  the  min¬ 
imum  set  of  privileges  required. 

4.  MILS  has  been  proposed  as  an  approach 
to  building  secure  systems  [9, 10].  MILS 
is  a  joint  research  effort  of  academia, 
industry,  and  government,  led  by  the 
U.S.  Ar  Force  Research  Laboratory.  The 
MILS  approach  is  based  on  the  notion 
of  separating  —  and  thus  limiting  the 
scope  and  reducing  the  complexity  of  — 
the  security  mechanisms. 


®  The  Architecture  Tradeoff  Analysis  Method  is  registered  in 
the  U.S.  Patent  and  Trademark  Office  by  Carnegie  Mellon 
University. 


Jorgen  Hansson,  Ph.D., 

is  a  senior  member  of  the 
technical  staff  and  lead 
of  the  Performance-Crit¬ 
ical  Systems  (PCS)  Initia¬ 
tive  at  the  SEI,  and  is  a 
professor  of  computer  science  at  Lin- 
koping  University,  Sweden.  The  Initia¬ 
tive  focuses  on  developing,  maturing, 
and  transitioning  analysis-based  assur¬ 
ance  and  model-based  engineering  tools 
and  practices  for  predicting  the  depend¬ 
ability  and  performance  of  software  sys¬ 
tems.  Hansson  holds  bachelor’s  and 
master’s  degrees  in  computer  science 
from  University  of  Skovde,  Sweden,  and 
a  doctorate  in  computer  science  from 
Linkoping  University,  Sweden. 

SEI 

4500  Fifth  AVE 
Pittsburgh,  PA  15213 
Phone:  (412)  268-6733 
Fax:  (412)  268-5758 
E-mail:  hansson@sei.cmu.edu 


About  the  Authors 

Peter  H.  FeUer,  Ph.D., 

is  a  senior  member  of 
the  technical  staff  in  the 
PCS  Initiative  at  the  SEI. 
He  has  authored  more 
than  80  articles  in  the 
areas  of  dependable  real-time  systems, 
architecture  languages  for  embedded 
systems,  and  predictable  system  analysis 
and  engineering.  Feiler  is  the  technical 
lead  and  author  of  the  SAE  AS-2C 
AADL  standard.  Feiler  holds  a  Vordi- 
plom  degree  in  math/computer  science 
from  Technical  University  Munich,  and  a 
doctorate  in  computer  science  from 
Carnegie  Mellon  University. 

SEI 

4500  Fifth  AVE 
Pittsburgh,  PA  15213 
Phone:  (412)  268-7790 
Fax:  (4 1 2)  268-5758 
E-mail:  phf@sei.cmu.edu 


John  Morley  is  a  mem¬ 
ber  of  the  operating  staff 
at  the  SEI.  He  has 
reported  on  model-based 
engineering  and  service- 
oriented  architecture  for 
SEI  publications,  has  more  than  20  years 
experience  in  writing  and  editing  scien¬ 
tific  and  technical  materials,  and  holds  a 
master’s  degree  in  English  literature 
from  Duquesne  University. 

SEI 

4500  Fifth  AVE 
Pittsburgh,  PA  15213 
Phone:  (4 1 2)  268-6599 
Fax:  (412)  268-5758 
E-mail:  jmorley@sei.cmu.edu 


I  4  Crosstalk  The  Journal  of  Defense  Software  Engineering 


September  2008 


Practical  Defense  in  Depth 


Michael  Howard 
Microsoft  Corp 

As  part  of  its  ongoing  commitment  to  Bill  Gates’  vision  of  Trustworthy  Computing,  Microsoft  officially  adopted  important 
security-  and  privacy-related  disciplines  to  its  software  development  process.  These  changes,  called  the  Security  Tdevelopment 
Tifecycle  (SDL)  have  led  to  a  demonstrable  reduction  in  security  vulnerabilities  in  products  such  as  Microsoft’s  Windows 
Vista  operating  system  and  its  SQL  Server  2005  database.  The  purpose  of  this  article  is  not  to  describe  the  SDL  in  detail, 
but  to  outline  some  of  the  practical  defensive  measurements  in  use  at  Microsoft  required  by  the  SDL.  If  Microsoft’s  SDL 
is  new  to  you,  refer  to  the  page  16  sidebar,  “A  Brief  SDL  Overview.” 


Even  though  the  vulnerability  counts 
have  dropped,  the  number  of  vulner¬ 
abilities  is  not  zero.  And,  even  in  my 
wildest  dreams,  I  do  not  think  we  will  get 
to  zero.  I  will  explain  why  shortly. 

In  the  very  early  days  of  the  SDL, 
Microsoft  focused  heavily  on  removing 
design  and  code-level  security  vulnerabili¬ 
ties;  as  we  progressed,  we  added  processes 
that  help  reduce  the  chance  that  new  vul¬ 
nerabilities  get  added  to  the  software. 

Examples  of  implementation  require¬ 
ments  in  the  SDL  include: 

•  Use  of  code  analysis  tools  on  develop¬ 
er’s  desktops  to  find  security  vulnera¬ 
bilities. 

•  Removing  known  insecure  functions 
(such  as  the  C  runtime  strcpy  and  strncpy 
functions) . 

•  Migrating  weak  cryptographic  algo¬ 
rithms  to  more  robust  algorithms 
(such  as  Data  Encryption  Standard  to 
Advanced  Encryption  Standard, 
Secure  High  Algorithm  (SHA)-1  to 
SHA-256). 

But  the  SDL  is  constantly  evolving.  We 
update  the  SDL  roughly  twice  a  year  so  we 
can  keep  pace  with  new  vulnerabilities  and 
new  security  research.  We  find  the  contin¬ 
uous  improvement  to  the  SDL  to  be  a  sig¬ 
nificant  benefit  compared  to  static  securi¬ 
ty  certification  programs  —  such  as  the 
Common  Criteria  —  which  do  not  evolve 
so  quickly. 

Over  the  last  few  years,  the  SDL  has 
been  extended  to  embrace  a  stronger 
focus  on  defense  in  depth.  While  some 
updates  to  the  SDL  continue  to  address 
design  and  implementation  vulnerabilities, 
more  of  today’s  SDL  requirements  focus 
on  defense  in  depth. 

The  prime  driver  for  this  change  is  a 
realization  that  you  can  never  remove  all 
security  vulnerabilities  from  software.  The 
reason  this  statement  is  true  is  simple. 
Some  software  you  create  might  be  com¬ 
pletely  secure  today,  but  that  could  all 
change  tomorrow  when  security 
researchers  learn  (and  potentially  make 


public)  new  classes  of  vulnerabilities. 

Allow  me  to  illustrate  the  point  with  a 
real  example. 

In  October  2003,  Microsoft  issued  a 
security  bulletin,  MS03-047  [1]  that  fixed  a 
cross-site  scripting  (XSS)  vulnerability  in 
the  Outlook  Web  Access  (OWA)  front 
end  to  Microsoft’s  Exchange  5.5  software. 
In  August  2004,  Microsoft  issued  another 


*^Some  software  you 
create  might  be 
completely  secure  today, 
but  that  could  all  change 
tomorrow  when  security 
researchers  learn  (and 
potentially  make  public) 
new  classes  of 
vulnerabilities/^ 


bulletin,  MS04-026  [2],  in  the  same  OWA 
component  that  was  fixed  in  MS03-047  to 
fix  an  XSS  variation  called  HTTP 
response  splitting.  Interestingly,  the  code 
fixes  were  relatively  close  to  one  another. 
So  the  question  that’s  probably  on  your 
mind  is,  “What  happened?  How  did  you 
guys  miss  the  bug  that  led  to  MS04-026?” 
The  answer  is  simple.  At  the  time  we 
issued  MS03-047,  the  world  had  not  heard 
of  HTTP  response  splitting  vulnerabili¬ 
ties.  In  theory,  the  Microsoft  Exchange 
engineers  could  have  scoured  the  code 
and  fixed  every  known  security  bug,  but 
they  would  probably  have  missed  the  vul¬ 
nerability  that  led  to  MS04-026  because 
nobody  knew  of  that  class  of  vulnerabili¬ 
ty  at  the  time.  So  what  changed?  What  led 
to  the  security  vulnerability?  In  March 


2004,  Sanctum  (purchased  by  Watchfire, 
which  has  since  been  purchased  by  IBM) 
released  a  paper  entitled  “Divide  and 
Conquer”  [3]  describing  a  variation  of  the 
XSS  vulnerability.  When  the  Microsoft 
engineers  fixed  the  first  bug,  the  second 
class  of  bug  was  unheard  of 

Unfortunately,  there  were  no  defense 
in  depth  mechanisms  in  place  to  protect 
customers  from  either  of  these  vulnerabil¬ 
ities,  so  customers  had  to  apply  a  security 
update  to  protect  themselves. 

Another  example  is  the  integer  arith¬ 
metic  vulnerability  [4].  Without  going  into 
a  detailed  explanation,  this  kind  of  bug 
was  unheard  of  10  years  ago,  and  is  a  very 
common  security  vulnerability  today. 

The  moral  of  this  story  is  that  you  can 
never  create  totally  secure  code.  There  are 
many  reasons  why: 

•  People  make  mistakes. 

•  Tools  are  not  perfect. 

•  Security  can  have  very  subtle  nuances 
that  only  security  experts  understand, 
especially  with  regard  to  cryptography. 

•  The  Internet  is  an  asymmetric  battle¬ 
ground.  There  are  many  hidden  and 
skilled  attackers  who  can  strike  at  will, 
but  defenders  must  be  constantly  vigi¬ 
lant  and  never  make  mistakes. 

•  Most  importantly,  we  cannot  predict 
future  classes  of  vulnerabilities. 

There  are  two  final  points  that  I  really 

wish  to  stress  because  these  make  defense 
in  depth  especially  critical  today: 

1.  As  the  security  vulnerability  landscape 
evolves,  more  vulnerability  informa¬ 
tion  is  moving  underground  and  being 
used  by  criminals  to  attack  sensitive 
systems.  The  defenders  do  not  know 
the  system  is  vulnerable,  so  no  securi¬ 
ty  update  is  available.  This  is  clearly  a 
risk  for  government  systems. 

2.  Somewhat  similar  to  the  first  point  is 
that  we  are  seeing  more  zero-day  vul¬ 
nerabilities.  There  may  be  no  attacks 
yet,  but  there  is  no  update  or 
workaround  either. 

The  rest  of  this  article  outlines  some 
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A  Brief  SDL  Overview 

The  SDL  is  a  set  of  requirements  and  recommendations  added  to  an  existing  soft¬ 
ware  development  process  to  improve  security.  A  side  benefit  of  the  SDL  is  increased 
robustness  since  many  security  vulnerabilities  also  affect  robustness.  SDL  is  all  about 
security  improvement,  not  perfection. 

There  are  two  goals  to  the  SDL.  The  first  is  to  reduce  the  number  of  security  vul¬ 
nerabilities  in  Microsoft  products.  This  is  done  by  removing  vulnerabilities  from  software, 
or  better  yet,  not  adding  vulnerabilities  to  the  code  from  the  outset.  This  reduction  in  vul¬ 
nerabilities  is  achieved  through  education,  tooling,  better  libraries,  and  so  forth.  The  sec¬ 
ond  goal  is  to  reduce  the  severity  of  any  vulnerabilities  that  are  inadvertently  left  in  the 
product.  You  can  reduce  severity  by  adding  defensive  mechanisms.  The  purpose  of  this 
article  is  to  explain  some  of  those  defenses. 

In  a  nutshell,  we  teach  people  to:  Do  everything  possible  to  make  your  product  as 
secure  as  possible,  but  assume  it  will  fail. 

A  requirement  defines  an  SDL  task  that  must  be  completed  prior  to  giving  the  code 
to  customers,  and  a  recommendation  defines  a  best  practice  that  should  be  considered 
by  the  development  team.  It  is  not  uncommon  for  a  security  best  practice  to  start  out  as 
a  recommendation  and  then  progress  to  a  requirement. 

The  SDL  defines  requirements  and  recommendations  for: 

•  Education. 

•  Risk  assessment. 

•  Threat  modeling. 

•  Coding. 

•  Testing. 

•  Final  security  review. 

•  Maintenance. 

You  can  learn  more  about  the  SDL  in  “The  Security  Development  Lifecycle” 
(Microsoft  Press,  Howard  and  Lipner),  or  at  the  SDL  blog:  <http://blogs.msdn.com/sdl>. 


of  the  defense  in  depth  techniques  and 
technologies  applied  as  part  of  the  SDL  at 
Microsoft. 

Classes  of  Defense  in  Depth 
Mechanisms 

Under  the  SDL,  there  are  two  distinct 
types  of  defense  in  depth  mechanisms.  We 
don’t  call  them  out  expUdtly  as  different 
classes,  but  the  distinction  between  the 
two  classes  is  understood. 

The  first  type  of  defense  is  designed  to 
totally  stop  an  attacker  from  accessing  a 
system  or  software.  This  class  of  defense 
offers  a  level  of  assurance  that  stops  an 
attacker  when  the  defense  is  correctly  con¬ 
figured  and  used.  If  it  does  not  stop  an 
attacker,  then  the  defense  has  a  vulnerabil¬ 
ity  that  must  be  fixed.  One  example  is  the 
ubiquitous  firewall.  The  SDL  sets  strict 
requirements  on  the  process  for  opening 
an  inbound  port  on  the  Windows  Firewall. 
Other  examples  include  permissions  on 
objects:  a  weak  permission  could  render  a 
system  insecure.  For  example,  Microsoft 
issued  a  security  bulletin,  MS04-005  [5], 
for  VirtualPC  for  the  Apple  Macintosh 
because  of  a  weak  permission  on  a  critical 
file  that  led  to  a  symUnk-style  attack  [4]. 

Strong  operating  system  access  con¬ 
trols  are  an  important  part  of  the  SDL 
and  critically  important  to  any  system; 
however,  a  new  requirement  in  the  SDL 
this  past  year  is  strong  access  control 
mechanisms  on  database  objects. 


Structured  Query  Language  (SQL)  injec¬ 
tion  vulnerabilities  are  a  well  understood 
vulnerability  that  can  lead  to  disclosure  of 
sensitive  data,  data  corruption,  and,  in 
some  cases,  system  compromise  [4].  The 
industry  as  a  whole  has  created  best  prac¬ 
tice  documentation  and  tools  to  help 
remove  these  critical  bugs,  but  all  it  takes 
is  a  new  attack  type  or  an  error  to  leave  a 
customer  open  to  attack  and  compromise. 
The  SDL-required  defense  in  depth  mech¬ 
anisms  help  to  mitigate  this  risk.  In  short, 
direct  access  to  the  underlying  tables  is 
explicitly  denied  to  all  but  the  database 
administrators,  and  untrusted  access  is 
limited  to  appropriate  database  objects 
such  as  stored  procedures  and  views. 
These  objects  are  granted  access  to  the 
underlying  data.  This  configuration  has 
the  effect  that  if  an  attacker  can  bypass 
the  normal  defenses  against  SQL  injec¬ 
tion,  he  or  she  still  cannot  read  the  under¬ 
lying  data  in  the  tables.  The  correct  reme¬ 
dy  to  prevent  SQL  injection  is  to  build 
safe  SQL  queries,  but  the  table-level 
defense  is  there  solely  in  case  the  remedy 
fails  or  is  implemented  incorrectly. 

The  second  type  of  defense  is  a  set  of 
mechanisms  that  is  designed  to  slow  an 
attacker  down  or  make  an  attacker  create  a 
different  exploit  to  attack  a  system.  I  want 
to  spend  most  of  this  article  on  this  sub¬ 
ject.  At  Microsoft,  we  continue  to  spend  a 
great  deal  of  time  and  effort  researching, 
designing,  and  implementing  these  defens¬ 


es:  most  of  them  are  intended  to  help  mit¬ 
igate  buffer  overrun  and  integer  overflow 
vulnerabilities.  A  great  deal  of  C  and  C-F-F 
code  exists  today,  and  even  more  is  written 
every  day.  In  a  perfect  world,  people 
would  simply  abandon  C  and  C-F-F  in 
favor  of  safer  programming  languages 
such  as  C#  or  Java,  but  in  our  imperfect 
world,  C  and  C-F-F  are  often  the  correct 
tools  for  the  job.  Again,  in  a  perfect  world, 
C  and  C-F-F  developers  would  write  secure 
code,  but  in  our  imperfect  world  this  is 
not  always  possible;  however,  it  is  impor¬ 
tant  that  C  and  C-F-F  developers  take 
advantage  of  defense  in  depth  mecha¬ 
nisms.  The  SDL  mandates  a  number  of 
important  C  and  C-F-F  defenses,  including: 

•  Address  randomization. 

•  Stack-based  buffer  overrun  detection. 

•  Heap  corruption  detection. 

•  Pointer  protection. 

•  No-execute  (often  called  NX  or 

W^X). 

•  Service  failure  restart  policy. 

Note  that  none  of  these  defenses  actu¬ 
ally  remove  vulnerabilities,  nor  do  they 
magically  make  software  more  secure. 
What  they  do  is  turn  a  potential  code  exe¬ 
cution  exploit  into  a  denial-of-service  bug 
because  these  defenses  will  simply  fail  the 
application  if  they  detect  an  anomalous 
condition.  If  an  application  crashes,  it  also 
gives  the  attacker  fewer  opportunities  to 
re-attempt  an  attack. 

I  do  not  intend  to  cover  each  of  these 
in  deep  technical  detail.  The  reader  is 
urged  to  e-mail  the  author  or  refer  to  the 
references  for  further  information  [6,  7]. 

Address  Randomization 

There  is  nothing  attackers  love  more  than 
a  predictable  system  since  it  makes  build¬ 
ing  reliable  exploits  easier.  Reliable 
exploits  are  harder  to  detect  because  they 
usually  execute  correctly  and  do  not  crash 
or  alert  the  system  operators  to  nefarious 
acts.  Windows  Vista  and  Windows  Server 
2008  (and  later)  offer  image  randomiza¬ 
tion,  stack  randomization,  and  heap  ran¬ 
domization.  Image  randomization  relo¬ 
cates  the  entire  operating  system  into  one 
of  256  possible  configurations  on  each 
reboot.  By  default,  non-operating  system 
images  are  not  randomized,  and  third- 
party  components  must  opt  into  image 
randomization  using  the  /DYNAM- 
ICBASE  linker  option  in  Visual  C-F-F 
2005  Service  Pack  1  and  later.  Some  ver¬ 
sions  of  gnu’s  C  Compiler  (GCC)  and 
some  versions  of  Linux  and  Berkeley 
Software  Distribution  (BSD) -based  sys¬ 
tems  also  support  image  randomization  by 
using  the  -pie  compiler  option. 

Windows  Vista  and  Windows  Server 
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2008  and  later  also  support  stack  random¬ 
ization;  when  a  thread  is  started,  the 
thread’s  stack  is  offset  by  up  to  32  pages  (4 
kilobytes  on  a  32-bit  central  processing 
unit).  Again,  this  option  is  available  by 
Hnking  with  /DYNAMICBASE. 

Finally,  Windows  Vista  and  Windows 
Server  2008  (and  later)  support  heap  ran¬ 
domization,  meaning  that  when  an  appli¬ 
cation  allocates  dynamic  memory  from 
the  system  heap,  the  operating  system  off¬ 
sets  the  start  of  the  heap  by  a  random 
amount.  Heap  randomization  is  enabled 
by  default  in  Windows  Vista  and  Windows 
Server  2008. 

Together,  image,  stack,  and  heap  ran¬ 
domization  can  seriously  hinder  an  attack¬ 
er  dealing  with  the  lack  of  predictability. 
Usually  the  sign  of  a  failed  attack  is  a  crash 
in  the  application  under  attack.  This 
means  that  system  administrators  should 
really  pay  attention  to  applications  that 
crash,  as  crashes  may  not  just  be  the  sign 
of  a  coding  bug  but  a  sign  of  a  security 
bug  under  attack. 

Stack-Based  Buffer  Overrun 
Detection 

Stack-based  buffer  overrun  detection  is 
available  in  Microsoft  Visual  C++  2003 
and  later  through  the  /GS  compiler 
switch.  It  works  by  adding  a  random  num¬ 
ber  into  a  function’s  stack  frame  at  call 
time  and  when  the  function  returns  code 
inserted  by  the  compiler,  it  verifies  that  the 
random  number  has  not  changed.  If  it  has 
changed,  then  the  application  crashes 
because  a  stack-based  buffer  overrun  has 
been  detected.  At  this  point,  we  can  no 
longer  trust  the  integrity  of  the  data  or  the 
application.  Some  versions  of  the  GCC 
offer  a  similar  defense  by  using  -fstack- 
protector. 

This  defense  could  require  the  attacker 
to  build  a  specific  attack  to  circumvent  the 
defense.  A  good  example  of  how  this 
helped  protect  customer  is  the  Blaster 
Worm.  On  Windows  Server  2003  (but  not 
Windows  2000  or  Windows  XP),  the  vul¬ 
nerable  component  was  compiled  with 
/GS.  The  malicious  Blaster  payload  was 
not  aware  of  the  / GS  defense,  so  it 
crashed  Windows  Server  2003  machines 
rather  than  infecting  them  with  the  Blaster 
Worm. 

Heap  Corruption  Detection 

Heap  corruption  detection  is  an  operating 
system  defense  available  in  Windows  Vista 
and  Windows  Server  2008  (and  later).  It  is 
similar  in  principle  to  stack-based  buffer 
overrun  detection  but  detects  heap  meta¬ 
data  corruption.  Again,  if  the  heap  is  cor¬ 


rupted,  the  operating  system  can  shut 
down  the  application,  reducing  the  chance 
that  an  attacker  will  re-attempt  a  failed 
attack. 

Pointer  Protection 

C  and  C++  pointers  are  an  attack  vector; 
if  an  attacker  can  overwrite  a  long-lived 
pointer  in  memory,  he  or  she  can  poten¬ 
tially  compromise  a  computer  by  writing 
arbitrary  data  at  a  predictable  location. 
Windows  includes  functions  that  encode 
(XOR)  a  pointer  with  a  random  value,  and 
this  operation  must  be  reversed  success¬ 
fully  in  order  to  get  the  valid  pointer  value. 
In  other  words,  if  an  attacker  attempts  to 
overwrite  a  pointer,  he  or  she  must  over¬ 
write  it  with  a  value  that  survives  the  un¬ 
encoding  operation.  Clearly,  this  is  not 
impossible,  but  it  is  another  defense  the 
attacker  must  overcome.  The  application 
programming  interfaces  in  Windows  that 
perform  these  operations  are: 

•  EncodePointer  and  DecodePointer. 

•  EncodeSystemPointer  and  Decode 
SystemPointer. 

Note  that  GLIBC  v2.5  (and  later)  have 
a  similar  defense,  but  it’s  mainly  used  in¬ 
side  GLIBC  itself  to  protect  sepmp  point¬ 
ers.  The  functions,  defined  in  sysdep.h  are 
PTR_MANGLE  and  PTR_DEMANGLE. 

No-Execute  (Often  Called  W'^X) 

Microsoft  calls  no-execute  data  execution 
prevention  (DEP).  This  defense  marks 
pages  of  memory  as  Writeable  or 
Executabk,  but  not  both.  Essentially,  this 
makes  it  very  hard  for  an  attacker  to  run 
malicious  code  out  of  a  writeable  memory 
segment.  Most  CPUs  today  support  this 
capability.  It  is  by  no  means  a  perfect 
defense  and  DEP  requires  randomization 
to  be  effective  at  stopping  a  class  of 
attacks  known  as  return-to-libc  [8] . 

In  Windows,  you  can  link  with 
/NXCOMPAT  to  opt-in  for  DEP. 

Some  versions  of  BSD  and  Linux  sup¬ 
port  W^X  also,  but  the  compiler  and 
operating  system  support  is  not  consistent 
across  platforms. 

Service  Failure  Restart  Policy 

A  final  defense  in  Windows  Vista  and 
Windows  Server  2008  was  very  hard  to 
implement  without  sacrificing  reliability. 
Windows  uses  many  services,  akin  to  Unix 
daemons,  to  perform  critical  system  tasks. 
Services  are  usually  long-lived  processes 
that  start  when  a  system  starts.  In  many 
cases,  administrators  want  a  crashed  ser¬ 
vice  to  simply  restart.  This  policy  gives 
better  uptime  to  customers.  This  is  great 
for  reliability,  but  it  can  be  terrible  for 
security  because  it  means  that  an  attacker 


can  keep  trying  his  attacks  over  and  over 
until  they  succeed.  The  ability  to  retry  is 
especially  important  in  a  system  that 
implements  a  great  deal  of  randomization, 
such  as  Windows  Vista  and  Windows 
Server  2008. 

Windows  offers  the  ability  to  define  a 
policy  that  restarts  a  failed  service  no 
more  than  a  certain  number  of  times  or 
on  a  certain  schedule.  For  example,  an 
administrator  could  define  a  policy  that 
win  restart  a  process  10  times  within  24 
hours,  and  after  that  no  longer  allow  it  to 
restart  unless  an  administrator  physically 
restarts  the  service.  It  is  also  possible  to 
restart  a  process  indefinitely.  But  we  don’t 
want  to  give  attackers  the  ability  to  re-try 
their  attacks  indefinitely,  so  we  tightened 
up  the  restart  policy  for  many  highly 
exposed  system  services. 

For  example,  in  Windows  Server  2008, 
many  services,  including  the  Network 
Access  Protection  Agent,  are  set  to  restart 
twice,  and  then  no  longer  restart.  In  other 
words,  the  attacker  has  two  shots.  With  all 
the  randomization  in  place  in  Windows 
today,  this  makes  the  attacker’s  job  much 
more  difficult. 

The  Question  of  Least 
Privilege 

You  may  have  noticed  that  I  have  not 
mentioned  least  privilege  as  a  defense,  and 
I  left  it  out  on  purpose.  Clearly,  least  priv¬ 
ilege  is  an  important  defense,  but  it  is  nec¬ 
essarily  an  imperfect  wall  because  many 
products  have  had  and  will  continue  to 
have  local  escalation  of  privilege  vulnera¬ 
bilities.  Also,  least  privilege  does  not  miti¬ 
gate  many  information  disclosure  vulnera¬ 
bilities.  Malicious  code  running  as  a  nor¬ 
mal  user,  rather  than  an  administrator,  can 
still  access  data  accessible  by  the  user,  and 
that  data  could  include  sensitive  data  such 
as  passwords,  encryption  keys,  personal 
financial  information,  and  e-mail.  Within 
the  SDL  we  think  of  least  privilege  as  very 
important,  but  we  also  recognize  that  on  a 
normal  user’s  computer,  it  can  be  hard  to 
enforce  the  security  boundary  and  have  a 
usable  system.  A  good  example  of  this  is 
running  mobile  code  through  a  Web 
browser.  At  some  point,  a  user  will  proba¬ 
bly  visit  a  Web  site  that  requires  a  Java 
applet,  a  Flash  file,  or  perhaps  some  mul¬ 
timedia  experience  that  will  require  some 
mobile  code.  Installing  this  code  is  a  trust¬ 
ed  operation,  so  the  user  must  elevate  to 
an  account  that  can  install  the  code.  The 
process  of  elevating  can  lead  to  weakness¬ 
es  in  a  pure  least  privilege  environment. 
Of  course,  it  is  possible  to  utterly  lock  a 
system  down  in  such  a  way  that  it  is  very 
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difficult  for  a  user  to  elevate  at  all;  for 
some  installations  processing  sensitive 
information,  this  is  the  right  answer. 

Summary 

Writing  code  that  is  perfectly  secure  in  the 
long  term  is  not  possible;  new  attack  types 
appear  almost  weekly.  But  it  is  imperative 
that  systems  offer  a  degree  of  protection, 
even  in  the  face  of  new  classes  of  attacks 
and  design  and  coding  vulnerabilities.  This 
means  that  software  development  organi¬ 
zations  should  spend  a  great  deal  of  time 
thinking  about  defense  in  depth  mecha¬ 
nisms,  as  well  as  focusing  on  “getting  the 
code  right.”  A  simple  mantra  to  consider 
is,  “Your  code  will  fail  -  now  what?” 

The  problem  is  exacerbated  by  zero- 
day  vulnerabilities,  and  vulnerability 
research  moving  underground  to  be  used 
for  criminal  purposes. 

If  there  is  one  lesson  we  can  all  learn, 
it  is  this:  Defense  in  depth  is  just  as  impor¬ 
tant  as  following  good  security  coding  and 
design  practices,  because  you  will  never 
get  the  product  totally  secure. 

If  there  is  a  second  lesson,  it  is  that 
you  must  use  as  many  defense  in  depth 
mechanisms  as  possible  and  they  must  be 
enabled  by  default  because  defense  in 
depth  is  most  useful  in  the  face  of  an 
attack  that  takes  advantage  of  a  vulnera¬ 
bility  that  is  not  publicly  known. 

We  have  implemented  the  defenses 
listed  above  in  various  Microsoft  products. 
I  would  urge  you  to  take  advantage  of 
these  defenses  if  you  build  on  the 
Microsoft  platform;  if  you  use  other  prod¬ 
ucts,  understand  what  defense  in  depth 
mechanisms  they  offer  and  use  them.^ 
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Supporting  Safe  Content-Inspection  of  Web  Traffic' 


Dr.  Partha  Pal  and  Michael  Atighetchi 
BBN  Technologies 

Interception  of  software  interaction  for  the  purpose  of  introducing  additional  functionality  or  alternative  behavior  is  a  well- 
known  software  engineering  technique  that  has  been  used  successfully  for  various  reasons,  including  security.  Software  wrap¬ 
pers,  firewalls,  Web  proxies,  and  a  number  of  middleware  constructs  all  depend  on  interception  to  achieve  their  respective  secu¬ 
rity,  fault  tolerance,  interoperability,  or  load  balancing  objectives.  Web  proxies,  as  used  by  organisations  to  monitor  and  secure 
Web  traffic  into  and  out  of  their  internal  networks,  provide  another  important  example. 


As  more  and  more  interactions  (includ¬ 
ing  personal,  financial,  and  social) 
become  Web-based,  a  number  of  observa¬ 
tions  can  be  made.  First,  as  technology  ad¬ 
vances  and  public  awareness  of  Internet 
security  increases,  an  increasing  portion  of 
Web  traffic  is  likely  to  be  carried  by 
Hypertext  Transfer  Protocol  Secure 
(HTTPS).  Second,  while  that  will  provide 
a  level  of  end-to-end  security,  it  will  pre¬ 
sent  a  new  challenge  for  the  functions  and 
services  that  rely  on  inspecting  the  con¬ 
tent  of  Web  traffic.  Some  of  these  ser¬ 
vices  and  functions  will  concern  security, 
such  as  auditing  and  access  control.  The 
challenge  comes  from  two  directions:  (1) 
the  standard  Web  proxies  of  today  pass 
the  HTTPS  traffic  through,  and  (2)  Web 
proxies  are  somewhat  global  (aggregating 
Web  traffic  from  many  users  or  applica¬ 
tions)  and  agnostic  to  personalization  to 
an  individual  user’s  or  an  application’s 
context  and  requirement.  We  developed  a 
personal  proxy  that  is  capable  of  handling 
both  HTTP  and  HTTPS  traffic,  and 
demonstrated  its  use  in  tackling  the  threat 
of  phishing  attacks.  This  personal  proxy 
win  be  a  useful  tool  for  implementing 
functions  and  services  that  require  inspec¬ 
tion  of  Web  traffic  content. 

Introduction 

The  ability  to  intercept  normal  interaction 
between  application  components  enabled 
a  number  of  useful  functions  such  as 
monitoring  and  auditing,  adaptive  failover, 
load  balancing,  and  (last  but  not  least) 
enforcement  of  security  policies. 
Obviously,  the  need  for  many  of  these 
functions  is  already  felt  in  the  context  of 
Web-based  applications.  The  use  of  Web 
proxies  by  organizations  to  monitor  and 
protect  Web-based  applications  running 
within  their  networks,  the  use  of  load  bal¬ 
ancing  mechanisms  in  server  farms,  and 
handling  cross-domain  exchanges  are 
cases  in  point. 

A  number  of  interception-based  func¬ 
tions  require  deep  inspection  of  the  traffic, 
meaning  operations  that  need  to  access 


the  content  of  the  payload,  not  just  the 
HTTP  header  information.  Web  proxies 
can  do  this  job  perfectly  for  HTTP  traffic, 
but  not  for  HTTPS  traffic.  The  reason  is 
that  HTTPS  is  the  secure  version  of  the 
HTTP  protocol,  and  HTTPS  payloads  are 
encrypted  by  Transport  Layer  Security  and 
are  not  meant  to  be  inspected  or  modified 
by  interlopers  like  the  proxy. 

As  important  services  increasingly 
become  Web-enabled  and  as  the  task  of 
setting  up  HTTPS  becomes  routine,  we 

'We  developed  a 
personal  proxy  that  is 
capable  of  handling  both 
HTTP  and  HTTPS  traffic, 
and  demonstrated  its  use 
in  tackling  the  threat  of 
phishing  attacks/* 

expect  that  increasing  Web  traffic  will 
move  over  to  HTTPS  to  provide  a  level  of 
security  that  the  users  have  come  to 
expect  (e.g.,  the  padlock  sign  on  the 
browser).  This  gain  in  one  aspect  of  secu¬ 
rity  (i.e.,  site  authentication  and  defense 
against  confidentiality  and  integrity  attacks 
on  the  information  during  the  transit) 
makes  it  difficult  for  functions  that  require 
access  to  the  content,  such  as  auditing  and 
monitoring,  application  level  rate  limiting, 
application  level  adaptive  caching,  con¬ 
text-specific  failover  and  load  balancing, 
and  so  forth.  In  addition,  as  Web  services 
become  the  de-facto  mechanism  of  infor¬ 
mation  exchange,  proxies  are  likely  to  play 
a  key  role  in  handling  cross-domain  issues. 
For  example,  opening  HTTP  connection 
to  Web  sites  other  than  the  one  from 
which  the  current  Web  page  was  served  is 
usually  not  permitted  from  the  browser. 


but  the  application  may  need  to  interact 
with  services  from  other  Web  sites.  Using 
a  proxy  is  one  solution  that  is  often  used 
to  get  around  that  problem.  The  problem 
gets  more  complicated  if  different  ser¬ 
vices  are  at  different  security  levels.  If  that 
transaction  happens  over  HTTPS,  the 
standard  proxies  will  be  of  no  use  -  one 
must  use  a  proxy  like  ours  that  can  proxy 
HTTPS. 

The  global  and  impersonal  nature  of 
the  proxies  poses  another  challenge. 
Unlike  a  firewall  (that  deals  with  many 
protocols  including  HTTP  and  many 
ports  including  those  used  by  Web  ser¬ 
vices),  a  Web  proxy  is  narrowly  focused 
on  the  HTTP  traffic.  However,  like  a  fire¬ 
wall,  a  Web  proxy  covers  multiple  hosts, 
users,  and  applications  in  an  aggregate 
form.  The  wide  variety  of  Web  applica¬ 
tions  and  their  range  of  importance  and 
sensitivity  —  from  financial  transactions 
like  banking  and  shopping  to  social  inter¬ 
actions  over  Facebook,  Web-based  e- 
mail,  and  chat  —  will  demand  an  unfore¬ 
seen  level  of  personalization  or  applica¬ 
tion-specificity  in  monitoring,  auditing, 
access  control,  rate  limiting,  or  load  bal¬ 
ancing  solutions.  We  claim  that  the  aggre¬ 
gate  and  one-size-fits-all  nature  of  Web 
proxies  will  make  the  Proxy-based 
Solutions  situated  at  the  Internet  Service 
Provider  (ISP)  or  at  corporate  boundaries 
insufficient  and  less  acceptable. 

On  one  hand,  the  users  will  be  less 
comfortable  disclosing  their  personal 
preferences  and  requirements  to  the 
remote  proxy  that  they  do  not  own  and 
control  themselves.  While  understanding 
and  enforcement  of  the  policy  may  be  a 
daunting  task  for  some  users,  they  will 
still  demand  canned  policies  that  they  can 
turn  on.  Think  of  setting  your  browser’s 
security  settings,  but  different  settings  for 
Facebook  and  your  bank,  and  even  dif¬ 
ferent  settings  for  different  Facebook 
users  in  your  household  that  you  can  con¬ 
trol.  Then,  there  will  always  be  a  group  of 
technology-literate  users  questioning  the 
adequacy  of  protection  of  personal  data 
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and  the  quality  of  enforcement  offered  at 
the  remote  proxy.  On  the  other  hand, 
because  the  remote  proxy  aggregates 
traffic  flow  from  multiple  users  and 
applications,  they  are  ill-equipped  to 
enforce  policies  and  preferences  that  are 
highly  specialized  (personalized)  for  indi¬ 
vidual  applications  or  users  without 
mutual  interference. 

We  argue  that  we  need  a  personal  Web 
proxy  that  will  do  the  following: 

•  Be  situated  near  the  user  or  the  appli¬ 
cation  it  proxies  (it  is  even  possible  to 
have  dedicated  proxies  for  each  appli¬ 
cation),  and  is  controlled  by  the  user 
or  the  owner  of  the  application  it  is 
proxying. 

•  Enforce  the  user’s  or  the  application’s 
policies  and  personal  preferences  that 
can  be  easily  plugged  in. 

•  Be  able  to  inspect  HTTPS  traffic  with¬ 
out  compromising  the  security  gains 
contributed  by  the  HTTPS  protocol. 
The  envisioned  personal  proxy  is 

analogous  to  personal  firewalls:  As  per¬ 
sonal  firewalls  bring  firewall  capability 
near  to  the  user’s  host  from  the  network 
edge,  the  personal  proxy  will  also  push 
proxying  capability  from  the  network 
edge  closer  to  the  user  or  the  application. 
Furthermore,  the  personal  proxy  is  a 
valid  application  level  proxying  mecha¬ 
nism  that  can  be  easily  customized  for 
the  application  or  user  at  hand;  it  pro¬ 
vides  an  easy  way  to  introduce  additional 
application  or  user-specific  functionality 
in  the  HTTP/HTTPS  path. 

There  are  a  number  of  software  engi¬ 
neering  reasons  supporting  the  need  of  a 
separate  proxy,  as  opposed  to  embedding 
the  needed  additional  functions  into  the 
application  components  it  mediates 
between.  First,  the  proxy  adds  a  separate 
layer  of  protection  (another  process  to 
corrupt  —  a  crumple  zone,  if  you  wH),  and 
provides  stronger  isolation  guarantees 
(defense  against  memory  corruption 
attacks)  and  increased  flexibility.  The  proxy 
is  less  complex  than  the  browser  that  has 
to  support  applications  ranging  from 
streaming  media  to  Java  applet,  and  pro¬ 
vides  a  smaller  attack  surface.  Since  the 
proxy  is  a  dedicated  process,  it  can  be  pro¬ 
tected  using  technologies  that  implement 
process  protection  domains,  such  as 
SELinux  [1]  or  Cisco  Security  Agent  [2]. 
Second,  a  personal  proxy  offers  a  good 
middle  ground  between  the  two  extremes, 
dealing  with  the  aggregate  of  interactions 
at  the  network  edge  or  modifying  each 
application.  A  browser  plugin-based 
implementation  will  not  be  able  to  control 
or  monitor  non-browser  applications  that 
may  use  HTTP  or  HTTPS  and  should  be 


subject  to  the  same  user-defined  policies 
and  preferences.  To  cover  this  situation, 
one  either  assumes  (somewhat  unrealisti¬ 
cally)  that  all  applications  interacting  over 
HTTP/HTTPS  use  the  browser  or  are 
forced  to  develop  similar  embedded  capa¬ 
bilities  for  each  of  those  non-browser 
applications.  Furthermore,  the  corporate 
or  ISP  proxy  may  not  be  able  to  enforce 
policies  of  individual  applications  and 
users  at  the  network  edge.  It  is  easier  to 
implement  user-  or  application-specific 
policies  and  behavior  into  a  personal  proxy 
that  runs  on  the  user’s  host  and,  using  fire¬ 
wall  rules,  mandate  that  the  only  way 
HTTP/HTTPS  traffic  gets  in  or  out  is 
through  the  proxy.  Third,  any  mechanism 
that  enables  flexible  and  customizable 
introduction  of  additional  behavior,  con¬ 
straint  enforcement,  and  monitoring  with¬ 
out  requiring  costly  (and  sometimes 
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impossible)  code  changes  in  the  original 
application  is  a  valuable  software  engineer¬ 
ing  asset  The  personal  proxy  performs 
this  job  adequately.  Other  than  ensuring 
that  the  HTTP/HTTPS  traffic  flows 
through  it,  no  code  change  is  necessary  for 
the  applications  that  interact  through  it. 
Finally,  to  be  general  and  to  support  aU 
kinds  of  monitoring  and  inspection  use- 
cases,  the  additional  user-  or  application- 
specific  policies  and  behavior  must  be 
inserted  before  traffic  is  encrypted  with 
the  remote  site’s  key.  To  illustrate  the 
point,  note  that  Chinese  users  are  able  to 
bypass  governmental  scrutiny  enforced  at 
their  network  edge  by  interacting  with 
encrypting  proxies  outside  China.  While 
our  proposed  personal  proxies  are  con¬ 
trolled  by  the  user/ application  it  covers  (as 
opposed  to  any  government  agency),  there 
are  use-cases  (e.g.,  parental  control,  cross¬ 
domain  security  policy  enforcement) 
where  personal  proxies  provide  a  better 
solution  than  the  browser-embedded 
checks  or  proxies  at  the  edge. 

Under  Department  of  Homeland 
Security  (DHS)  funding,  we  have  devel¬ 
oped  a  customizable  Web  proxy  that  han¬ 
dles  both  HTTP  and  HTTPS  protocols. 
For  HTTPS,  the  proxy  works  by  establish¬ 
ing  two  System  Specification  Language 


(SSL)  connections:  one  between  the 
browser  and  the  proxy,  and  the  other 
between  the  proxy  and  the  remote  Web 
site.  The  customization  happens  by  con¬ 
figuring  the  proxy’s  chain  of  interceptors. 
The  proxy  can  be  placed  near  the  user,  on 
the  user’s  computer,  or  at  the  user’s  home 
router  box.  We  have  demonstrated  how 
such  a  personalized  proxy  can  be  used  to 
protect  the  user  from  divulging  personal 
information  to  malicious  Web  sites  (i.e., 
defense  against  phishing  attacks).  We  have 
started  investigating  other  uses  of  the 
proxy,  such  as  auditing  inter-agent  com¬ 
munication  in  a  semantic  Web  application 
so  that  the  recorded  interactions  can  be 
used  by  machine-learning  algorithms  that 
aim  to  learn  and  improve  how  the  agents 
achieve  their  tasks.  In  this  article,  we 
briefly  describe  the  architecture  and  oper¬ 
ation  of  this  personal  proxy;  a  detailed 
description  and  the  anti-phishing  applica¬ 
tion  appears  in  [3]. 

Architecture  of  the  Personal 
Proxy 

Figure  1  illustrates  the  design  of  the  per¬ 
sonal  proxy,  which  consists  of  four  main 
modules  that  are  implemented  on  top  of 
Jetty,  a  popular  open-source  Web  server 
written  in  Java  [4].  The  plugin  framework 
provides  a  means  for  integration  of  cus¬ 
tom  reactive  and  proactive  behavior.  In 
the  first  application  of  this  proxy,  all  anti¬ 
phishing  checks  were  implemented  as  a  set 
of  plugins  for  this  module.  A  plugin  can 
be  one  of  the  following  three  types, 
depending  on  its  role  in  the  overall  control 
flow  and  threading  logic: 

•  Data  plugins.  Each  data  plugin  is 
invoked  on  every  request  and  associat¬ 
ed  response.  A  data  plugin  is  used  for 
handling  the  header  and  payload  data 
based  on  a  specified  security  policy. 
For  example,  a  proxy  could  be  config¬ 
ured  to  record  all  or  selected  parts  of 
Web  traffic  as  part  of  a  parental  con¬ 
trol  policy.  Recordings  can  be  persisted 
securely  on  the  disk. 

•  Checks.  These  plugins  are  organized 
in  a  chain,  and  intercepted  requests 
flow  through  these  checks  like  a 
pipeline.  An  individual  check  exits  with 
either  a  break  or  a  continue.  A  continue 
indicates  that  the  request  goes  to  the 
next  stage,  possibly  with  some  addi¬ 
tional  metadata  tagged  to  it.  Breaks  can 
be  of  two  kinds:  A  negative  break  indi¬ 
cates  that  the  request  is  to  be  blocked, 
while  a  positive  break  indicates  that  the 
request  is  to  be  accepted.  In  either 
case,  a  break  implies  that  the  rest  of 
the  pipeline  stages  are  not  executed. 
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This  semantics  of  checks  is  amenable 
to  modular  implementation  and  inte¬ 
gration  of  security  policies. 

•  Probes.  In  contrast  to  checks  and  data 
plugins,  which  only  execute  reactively 
when  triggered  by  requests  or  respons¬ 
es,  probes  allow  us  to  embed  proactive 
behavior  into  the  proxy.  Probes  con¬ 
tain  dedicated  threads  that  trigger 
monitoring  functions  at  regular  config¬ 
urable  intervals.  The  probes  can  be 
configured  to  visit  specified  URLs  and 
scheduled  intervals  to  collect  data  that 
is  relevant  for  the  security  policy  con¬ 
text.  For  example,  in  the  case  of 
defending  against  phishing  attacks,  the 
probes  were  used  to  check  for  changes 
in  an  Internet  Protocol  address  or 
security  credential  of  the  banks  or 
financial  sites  registered  by  the  user. 
The  lower  part  of  Figure  1  displays  the 
remaining  three  modules.  The  modules  act 
as  access  paths  into  the  proxy.  The  HTTP 
Pro>gi  listens  on  a  configurable  network 
port  (e.g.,  8080)  for  incoming  HTTP 
requests,  and  dispatches  the  requests  to  a 
main  handler  (InterceptHandler),  which  in 
turn  makes  strategic  use  of  the  plugins. 
This  flow  is  similar  in  the  case  of  the 
HTTPS  Pro>cf,  except  that  it  listens  on  a 
different  network  port  (8443)  and  uses  a 
custom  extension  of  the  InterceptHandler 
(called  SslProxyHandler)  that  intercepts 
HTTPS  connect  requests  and  facilitates 
subsequent  interception  of  all  HTTPS 
requests  in  that  session.  The  third  access 
path,  HTTPS  Requests,  is  for  manage¬ 
ment  of  the  proxy  through  an  administra¬ 
tion  console.  Management  functions 
include  changing  the  order  of  plugins  and 
their  respective  importance  weights  as  well 
as  customization  of  user-specific  data. 
The  administrative  interface  is  optional 
for  out-of-the-box  deployment,  where  the 
proxy  is  preconfigured  and  preloaded  with 
appropriate  plugins  that  enforce  the 
desired  policy.  We  do  not  anticipate  that 
the  internal  details  are  important  for  most 
of  the  users  (beyond  pointing  their  appli¬ 
cations  or  browsers  to  the  proxy).  The 
users  who  write  and  package  custom  poli¬ 
cies  for  different  users  and  applications 
win  need  to  know  the  details  of  plugins.  A 
better  policy  interface,  supporting  a  gener¬ 
ation  of  plugins  (which  can  be  added  to 
the  proxy  by  editing  a  configuration  file) 
from  higher-level  policy  specification,  and 
a  better  way  to  inspect  the  policies  encod¬ 
ed  in  existing  plugins,  is  part  of  our  future 
work.  Once  this  policy  interface  is  in 
place,  these  users  will  also  be  shielded 
from  the  internal  details  and  complexities 
of  the  plugin  architecture.  If  the  internal 
details  change  because  of  evolution  of  the 


Figure  1:  Functional  Architecture  of  the  Personal  Pro^g 


Jetty  code  base/Web  services  specifica¬ 
tion,  only  the  policy  interface  implementa¬ 
tion  wiU  need  to  change. 

Placement  Options 

The  standard  deployment  of  the  proxy  is 
on  the  end  user’s  computer.  Although  this 
puts  a  small  load  on  the  Central 
Processing  Unit  (CPU),  memory,  and  disk 
resources  on  the  end  system,  it  has  the 
benefit  of  putting  the  proxy  under  direct 
control  of  the  end-user.  Our  understand¬ 
ing  is  that  end-users  feel  uncomfortable 
with  disclosing  personal  and  sensitive 
information  (preferences,  policies)  to 
external  parties,  but  are  more  amenable  to 
providing  this  information  to  local  com¬ 
ponents  as  long  as  it  doesn’t  leave  their 
machine.  Since  many  end-users  own 
either  a  wireless  or  DSL  router  and  since 
these  devices  already  ship  with  Web  serv¬ 
er  capabilities,  we  investigated  deploying 
the  proxy  on  a  Linksys  WRT54G  wireless 
router  running  OpenWrt  [5].  Another 
option  is  to  run  the  proxy  on  a  home 
router,  which  has  the  benefits  of 
increased  security  through  stronger  isola¬ 
tion  from  a  potentially  virus-infected 
desktop,  and  a  new  value-add  for  router 
manufacturers.  On  the  downside,  the  very 
limited  CPU  and  memory  resources  of 
the  home  routers,  especially  wireless 
routers,  significantly  lowers  the  perfor¬ 
mance  of  the  proxy. 

Insertion  Into  HTTP(S)  Flow 

Insertion  of  the  proxy  into  the  non- 
encrypted  HTTP  client-server  path  is 
straightforward  and  involves  changing  the 
client  application’s  proxy  settings  (e.g.. 


HTTP  Web  browser) .  To  prevent  an  attack¬ 
er  from  replacing  the  proxy  setting  to  a 
proxy  of  his  own,  and  to  ensure  that  any 
application  using  HTTP/HTTPS  is  subject 
to  the  security  policy  enforced  by  the  per¬ 
sonal  proxy,  firewall  rules  should  be  set  to 
only  allow  outgoing  Web  traffic  through 
the  personal  proxy.  For  intercepting 
encrypted  requests  from  client  application 
that  uses  HTTPS,  the  client  application’s 
(such  as  the  browser’s)  proxy  settings  are 
changed  accordingly  to  redirect  requests  to 
personal  proxy’s  HTTPS  port.  However, 
describing  how  appropriate  security  associ¬ 
ations  are  established  is  slightly  more 
involved  (see  Figure  2,  next  page). 

In  a  regular  use  case  without  any 
HTTPS  proxy,  SSL  relies  on  a  Public  Key 
Infrastructure  for  connection  establish¬ 
ment  [6].  Following  a  general  description 
of  the  SSL  protocol,  the  client  issues  a 
connection  request  to  the  server,  which 
the  server  acknowledges  with  a  response 
containing  a  certificate  signed  by  a  certifi¬ 
cation  authority  (CA).  The  client  then  con¬ 
tinues  to  perform  a  set  of  checks  on  the 
server  certificate,  the  main  one  of  which  is 
to  verify  that  the  CA’s  signature  is  valid.  In 
most  cases,  SSL  transactions  essentially 
establish  a  unidirectional  trust  relationship 
between  the  browser  and  the  target  Web 
server  via  a  commonly  trusted  CA. 

With  the  proxy  in  the  mix,  the  proto¬ 
col  becomes  a  little  more  complex.  The 
proxy  takes  on  the  role  of  a  server  when 
communicating  with  the  browser  and  the 
role  of  a  browser  when  communicating 
with  the  target  Web  server.  This  requires 
the  proxy  to  dynamically  generate  X509 
certificates  for  each  Domain  Name 
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System  name  it  is  proxying^  certified  by  its 
own  CA’  (called  PB  CA  in  Figure  2). 
During  installation,  the  Web  browser’s 
(and  any  other  application’s  using  HTTPS) 
settings  are  configured  to  trust  signatures 
from  the  PB  CA.  As  a  result,  the  overall 
trust  relationship  between  browser  and 
target  Web  server  can  now  be  decom¬ 
posed  into  two  daisy-chained  relation¬ 
ships,  one  between  the  browser  to  the  per¬ 
sonal  proxy,  and  a  second  between  the 
personal  proxy  and  the  target  Web  server. 

Does  the  proxy  introduce  additional 
security  vulnerabilities  by  breaking  the 
end-to-end  encryption  between  browser 
and  Web  server?  The  answer  to  this  ques¬ 
tion  depends  on  the  relative  trustworthi¬ 
ness  of  the  proxy  compared  to  the  brows¬ 
er  and  target  Web  server  and  where  it  is 
deployed.  Consider  the  case  where  the 
user  does  not  use  a  personal  proxy,  but 
thinks  that  his  desktop  and  the  servers  he 
uses  are  more  secure  than  the  ISP  server 
through  which  he  uses  the  Internet.  The 
ISP  server  may  co-host  other  applications, 
and  if  it  does  not  have  the  latest  security 
patches  installed,  such  a  setup  would  sig¬ 
nificantly  lower  the  overall  security  of 
Web  transactions  flowing  through  it.  On 
the  other  hand,  if  the  personal  proxy  is 
co-located  with  the  Web  browser  on  the 
same  desktop,  we  would  expect  it  would 
be  more  difficult  for  attackers  to  subvert 
or  corrupt  the  Java-based  stand-alone 
proxy  process  (which  only  listens  on  local- 
host)  compared  to  a  C^--l-  Web  browser 
running  Javascript.  In  both  cases,  data  is 
never  sent  unencrypted  over  the  network, 
so  the  guarantees  provided  by  SSL  across 
host  boundaries  are  not  affected. 

Performance  Overhead 

Introduction  of  a  clearly  noticeable  delay 
presents  increased  resistance  to  adoption 


of  the  new  technology.  To  minimize  per¬ 
formance  impact,  we  implemented  the 
proxy  on  top  of  the  high  performance 
Jetty  Web  server  and  implemented  various 
optimizations  in  the  SSL  proxy  architec¬ 
ture  to  keep  request  latencies  (i.e.,  elapsed 
time  between  a  request  and  its  response) 
within  user  acceptable  levels.  In  this  sec¬ 
tion,  we  use  overhead  to  mean  the  increase 
in  request  latency  due  to  interception  of 
HTTPS  traffic  by  the  proxy. 

We  measured  the  overhead  in  a  lab  set¬ 
ting  by  visiting  HTTPS  sites  without  the 
proxy  and  with  the  proxy  configured  with 
a  number  of  anti-phishing  checks.  The 
mean  time  to  load  the  visited  pages 
through  the  proxy  was  twice  the  mean 
time  to  load  the  same  pages  without  the 
proxy  (excluding  any  user  interaction  like 
typing  a  password  for  both  cases). 
However,  the  variance  of  load  time  was 
comparable  to  the  mean  (not  surprising 
because  we  were  visiting  sites  on  the 
Internet),  and  even  an  overhead  of  rough¬ 
ly  100  percent  was  not  distinguishable 
from  the  noise  (as  noted  by  external  field 
testers,  the  delay  introduced  by  the  proxy 
does  not  noticeably  impact  the  user  Web 
surfing  experience).  Much  of  this  over¬ 
head  can  be  attributed  to  crypto  opera¬ 
tions  and  session  multiplexing  performed 
in  Java.  We  expect  the  plumbing  overhead 
to  stay  independent  of  the  policy  checks 
enforced  by  the  proxy. 

We  also  compared  the  round-trip 
latencies  between  an  auditing  configura¬ 
tion  (when  the  proxy  is  simply  record¬ 
ing)  and  a  policy  enforcing  configuration 
(loaded  with  anti-phishing  checks).  We 
found  that  the  two  distributions  are  not 
significantly  different  as  their  inter-quar- 
tile  ranges  overlap  to  a  large  extent 
(from  200  to  1,500  milliseconds  [ms]) 
and  both  distributions  have  a  large  num¬ 
ber  of  oudiers  (some  even  greater  than 


Figure  2:  Personal  Proxy  as  a  Trusted  Middleman 
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50,000  ms).  We  suspect  that  available 
network  bandwidth  to  the  external  Web 
sites  together  with  available  CPU 
resources  of  those  sites  have  the  biggest 
impact  on  round  trip  latencies,  which  is 
why  the  distributions  looked  similar. 

Related  Work 

Various  HTTP  and  HTTPS  proxy  imple¬ 
mentations  exist  for  debugging  purposes 
(Burp  proxy  [7],  Charles  proxy  [8])  and 
Web  filtering  (WebCleaner  [9],  Privoxy 
[10]).  There  are  also  a  number  of  com¬ 
mercial  network  layer  tools  (e.g,  eSafe’s 
Web  Threat  Analyzer  [11],  McAfee 
IntruShield  [12])  that  can  inspect  Web 
traffic,  including  HTTPS  that  work  at  the 
enterprise  layer.  In  many  cases,  these  are 
geared  for  regulatory  and  auditing  compli¬ 
ance,  the  DHS-funded  research  focused 
on  transparent  inspection  of  SSL  traffic 
exclusively  for  regulatory  purpose. 
However,  we  were  unable  to  find  a  proxy 
that  could  be  used  as  a  general  purpose 
middleware  construct  for  customized  user 
and  application-specific  policies. 

Conclusion 

We  have  been  developing  advanced  mid¬ 
dleware  technologies  that  enable  adaptive 
behavior,  quality  of  service  (QoS)  man¬ 
agement  and  QoS-based  adaptive  behav¬ 
ior  in  distributed  systems  over  the  past 
several  years  [13].  In  doing  so,  we  have 
developed  middleware  constructs  for  han¬ 
dling  different  styles  of  distributed  inter¬ 
action  (e.g,  distributed  objects,  pubUsh- 
subscribe,  group  communication)  over  a 
number  of  protocols  (e.g,  socket-based. 
Common  Object  Request  Broker  Archi¬ 
tecture  or  Remote  Method  Invocation). 
The  present  work  involving  HTTP  and 
HTTPS  interception  complements  that 
line  of  successful  work,  and  enables  us  to 
introduce  advanced  middleware  capability 
to  distributed  systems  that  use  these  pro¬ 
tocols.  The  concept  of  a  personal  proxy 
has  the  potential  to  fill  an  important  and 
emerging  gap  in  the  current  Web-based 
systems  architecture. 

However,  as  noted  earlier,  the  person¬ 
al  proxy  is  still  in  its  early  stages  —  we  only 
have  a  prototype  implementation  that  is 
demonstrated  with  anti-phishing  checks, 
and  have  just  begun  exploring  its  use  in 
other  contexts. 

A  number  of  software  engineering  and 
usability  issues  also  need  additional  work, 
including  an  easy  way  to  inspect  enforced 
policies  and  the  ability  to  define  policies  at 
a  higher  level  of  abstraction  that  can  be 
automatically  translated  into  executable 
code  that  can  be  integrated  into  the  plug¬ 


in  framework.  These  are  the  next  steps  we 

hope  to  tackle.  ♦ 
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2.  To  increase  generation  performance, 
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ficates. 

3.  Alternatively,  the  PB  CA  can  be  signed 
by  a  common  root  CA. 
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Enhancing  the  Development  Life  Cycle 
To  Produce  Secure  Software 

Karen  Mercedes  Goertzel 
Boo^  Alkn  Hamilton 


Over  the  past  decades,  efforts  to  enhance  software  development  life  cycle  (SDLC)  practices  have  been  shown  to  improve  soft¬ 
ware  qualify  reliabilif,  and fault-tolerance.  More  recently,  similar  strategies  to  improve  the  security  of  software  in  organisa¬ 
tions  such  as  Microsoft,  Oracle,  and  Motorola  have  resulted  in  software  products  with  less  vulnerabilities  and  greater  depend¬ 
ability,  trustworthiness,  and  resilience.  In  its  mission  to  improve  the  security  of  software  used  in  America’s  critical  infra¬ 
structure  and  information  systems,  the  Department  of  Homeland  Security’s  (DHS)  Software  Assurance  Brogram  has  spon¬ 
sored  the  creation  of  the  book  Enhancing  the  Development  Life  Cycle  to  Produce  Secure  Software^  a  source  of 
practical  information  intended  to  help  developers,  integrators,  and  testers  identify  and  systematically  apply  security  and  assur¬ 
ance  principles,  methodologies,  and  techniques  to  current  SDLC  practices,  and  thereby  increase  the  security  of  the  software 
that  results.  Unlike  the  numerous  other  books  on  secure  software  development.  Enhancing  the  Development  Life  Cycle 
does  not  espouse  any  specific  methodology,  process  model,  or  development  philosophy.  Instead  it  explains  the  essentials  of  what 
makes  software  secure,  and  takes  an  unbiased  look  at  the  numerous  security  principles  and  secure  development  methodologies, 
practices,  techniques,  and  tools  that  developers  are  finding  effective  for  developing  secure  software  —  information  that  readers 
can  leverage  in  defining  their  own  SDLC  security-enhancement  strategies. 


In  its  report  to  President  George  W 
Bush  entitled  “Cyber  Security:  A  Crisis 
of  Prioritization”  (February  2005),  the 
President’s  Information  Technology 
Advisory  Committee  summed  up  the 
problem  of  non-secure  software: 

Network  connectivity  provides 
“door-to-door”  transportation  for 
attackers,  but  vulnerabilities  in  the 
software  residing  in  computers 
substantially  compound  the  cyber 
security  problem....  Software 
development  is  not  yet  a  science  or 
a  rigorous  discipline,  and  the  devel¬ 
opment  process  by  and  large  is  not 
controlled  to  minimize  the  vulner¬ 
abilities  that  attackers  exploit. 
Today,  as  with  cancer,  vulnerable 
software  can  be  invaded  and  modi¬ 
fied  to  cause  damage  to  previously 
healthy  software,  and  infected  soft¬ 
ware  can  replicate  itself  and  be  car¬ 
ried  across  networks  to  cause  dam¬ 
age  in  other  systems. 

Like  cancer,  these  damaging 
processes  may  be  invisible  to  the 
lay  person  even  though  experts 
recognize  that  their  threat  is  grow¬ 
ing.  And  as  in  cancer,  both  preven¬ 
tive  actions  and  research  are  criti¬ 
cal,  the  former  to  minimize  dam¬ 
age  today  and  the  latter  to  establish 
a  foundation  of  knowledge  and 
capabilities  that  wiU  assist  the  cyber 
security  professionals  of  tomorrow 
to  reduce  risk  and  minimize  dam¬ 
age  for  the  long  term.  Vulnerabili¬ 


ties  in  software  that  are  introduced 
by  mistake  or  poor  practices  are  a 
serious  problem  today.  In  the 
future,  the  nation  may  face  an  even 
more  challenging  problem  as 
adversaries  -  both  foreign  and  do¬ 
mestic  —  become  increasingly  so¬ 
phisticated  in  their  ability  to  insert 
malicious  code  into  critical  soft¬ 
ware. 

Software  is  considered  “secure”  when 
it  exhibits  three  interrelated  properties: 

1.  Dependability.  The  software  executes 
correctly  and  predictably,  even  when 
confronted  with  malicious  or  anom¬ 
alous  inputs  or  stimuli. 

2.  Trustworthiness.  The  software  itself 
contains  no  malicious  logic  or  any 
flaws  or  anomalies  that  could  be 
exploited  or  targeted  as  vulnerabilities 
by  attackers. 

3.  Resilience.  When  the  software  is  able 
to  resist  most  attempted  attacks,  toler¬ 
ate  the  majority  of  those  it  cannot 
resist,  and  recover  with  minimal  dam¬ 
age  from  the  very  few  attacks  that  suc¬ 
ceed  (i.e.,  those  the  software  could  nei¬ 
ther  resist  nor  tolerate) . 

A  number  of  factors  influence  how  likely 
software  is  to  consistently  exhibit  these 
properties  under  all  conditions.  These 
include: 

•  The  programming  language(s),  libraries, 
and  development  tools  used  to  design, 
implement,  and  test  the  software,  and 
how  they  were  used. 

•  How  the  software’s  re-used,  commer¬ 
cial  off-the-shelf,  and  open  source 


components  were  evaluated,  selected, 
and  integrated. 

•  How  the  software’s  executables  were 
distributed,  deployed,  configured,  and 
sustained. 

•  The  security  protections  and  services 
provided  to  the  software  by  its  execu¬ 
tion  environment. 

•  The  practices  used  to  develop  the  software,  and 
the  principles  that  governed  those  practices. 
Experience  over  the  past  few  decades 

has  shown  that  enhancing  SDLC  practices 
with  the  objective  of  improving  software 
quality,  reliability,  and  fault-tolerance  does, 
in  fact,  result  in  software  that  is  higher  in 
quality,  or  more  reliable,  or  more  tolerant 
of  faults.  More  recently,  the  same  SDLC 
enhancement  approach  has  been  applied 
to  improve  the  security  of  software.  By 
adjusting  and,  in  some  cases  expanding, 
SDLC  activities  to  ensure  that  they  consis¬ 
tently  adhere  to  secure  specification, 
design,  coding,  integration,  testing, 
deployment,  and  sustainment  principles, 
organizations  such  as  Microsoft,  Oracle, 
Motorola,  Praxis  High  Integrity  Systems, 
and  a  growing  number  of  others,  have 
been  able  to  report  that  soon  after  doing 
so,  they  were  landing  vulnerabilities  and 
weaknesses  much  earlier  in  the  software’s 
life  cycle.  In  turn,  those  organizations 
were  able  to  eradicate  problems  at  a  much 
lower  cost  than  ever  before.  Moreover,  the 
organizations  that  institutionalized  repeat¬ 
ed  use  of  security-enhanced  SDLC  practices 
found  that,  over  time,  the  enhanced  prac¬ 
tices  became  second-nature  to  their  devel¬ 
opers,  and  fewer  and  fewer  vulnerabilities 
and  weaknesses  appeared  in  their  software 
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in  the  first  place.  Not  only  that,  but  they 
also  noted  that  their  software’s  depend¬ 
ability,  trustworthiness,  and  resilience  also 
improved. 

Enhancing  the  Development 
Life  Cycle  to  Produce  Secure 
Software 

This  year,  the  DHS  sponsored  the  revision 
of  its  2006  document,  “Security  in  the 
Software  life  Cycle:  Making  Development 
Processes  -  and  Software  Produced  by 
Them  —  More  Secure'.”  The  new  docu¬ 
ment,  redded  “Enhancing  the  Software 
life  Cycle  to  Produce  Secure  Software,” 
transforms  what  was  essentially  a  com¬ 
pendium  of  software  security  assurance 
concepts  and  overviews  of  methodologies, 
process  models,  sound  practices  (also 
known  as  best  practices),  and  supporting  tech¬ 
nologies  that,  when  used,  had  been  report¬ 
ed  by  their  advocates  to  produce  software 
that  is  more  secure  than  software  built  by 
more  traditional  methods  and  tools. 

By  contrast,  the  authors  of 
“Enhancing  the  Software  Life  Cycle”  have 
transformed  their  previous  survey  of  the 
software  assurance  domain  into  a  source 
of  practical  information  to  enable  devel¬ 
opers,  integrators,  and  testers  to  identify 
and  systematically  apply  security  and 
assurance  principles,  methodologies,  and 
techniques  to  enhance  their  current  SDLC 
practices.  The  revision’s  focus  has  been 
narrowed  and  its  concept  discussions 
streamlined,  while  its  pragmatic  technical 
content  has  been  expanded  and  deepened 
and  augmented  with  extensive  lists  of  ref¬ 
erences  to  information  (online  and  in 
print)  on  how  to  implement  the  various 
techniques  and  methodologies  described 
in  the  document.  The  new  version  also 
reflects  pertinent  technological,  method¬ 
ological,  and  philosophical  advances  that 
have  occurred  in  the  software  and  soft¬ 
ware  assurance  domain  since  the  release  of 
“Security  in  the  Software  Life  Cycle”  more 
than  two  years  ago. 

Several  other  developments  made  this 
revision  possible.  The  Department  of 
Defense  (DoD)  Technical  Information 
Center  (DTIC)  sponsored  the  public 
release  of  “Software  Security  Assurance: 
A  State-of-the-Art  Report,”  which  cap¬ 
tured  and  updated  much  of  the  survey- 
type  information  in  “Security  in  the 
Software  Life  Cycle.”  In  addition,  DHS 
produced  its  drafts  of  “Software 
Assurance  in  Acquisition^,”  and  “Practical 
Measurement  Guidance  for  Software 
Assurance  and  Information  Security’.” 
The  DoD  drafted  “Engineering  for 
System  Assurance”'',  the  DTIC-sponsored 


“Project  Management  for  Software 
Assurance:  A  State-of-the-Art  Report,”’ 
and  Addison-Wesley  published  “Software 
Security  Engineering:  A  Guide  for  Project 
Managers”  as  part  of  its  Software  Security 
Series’.  Collectively,  these  publications 
address  many  concerns  of  the  secondary 
intended  audience  for  “Security  in  the 
Software  Life  Cycle,”  including  acquisition 
managers,  project  managers,  system  engi¬ 
neers,  and  information  security  practition¬ 
ers,  enabling  “Enhancing  the  Software 
Development  Life  Cycle”  to  focus  on  its 
primary  audience  of  developers,  integra¬ 
tors,  and  testers. 

Unlike  the  steadily  increasing  number 
of  other  books  on  secure  software  devel¬ 
opment,  secure  programming,  application 
security,  and  software  security  testing, 
“Enhancing  the  Development  Life  Cycle” 
strives  to  remain  methodology  j process-agnostic. 
Its  intent  is  to  explain  the  essentials  and 
characterize  the  advantages  of  a  number 
of  security  principles,  and  secure  develop¬ 
ment  methodologies,  practices,  and  tech¬ 
niques  that  have  proven  effective  in  the 
security-enhancement  of  SDLC  activities. 

“Enhancing  the  Development  Life 
Cycle”  is  currently  undergoing  a  final 
review  by  the  DHS/DoD  co-sponsored 
Software  Assurance  Working  Groups. 
Based  on  comments  from  those  reviewers, 
the  final  draft  will  be  produced  for  public 
comment  late  this  summer.  It  will  be  avail¬ 
able  for  download  from  the  Build  Security 
In  Web  site  at  <https:/ /buildsecutityin.us 
-cert.gov/ daisy/bsi/ dhs.html>.^ 
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Corey  P.  Cunha 
Savard  Engineering 

Developing  safety-critical  software  is  often  an  extremely  complicated  process,  and  if  managed  incorrectly  could  have  the  ten¬ 
dency  to  cause  more  harm  than  good.  In  order  to  deal  with  the  challenge  of  writing  safety-critical  software,  certain  considera¬ 
tions  must  be  followed.  Different  case  studies  will  be  used  in  this  article  to  illustrate  points  about  the  ethics  standards,  hasy 
ard  identification  challenges,  and  aftermath  management  techniques  needed  to  effectively  manage  the  develepment  and  deploy¬ 
ment  of  safety-critical  software. 


Safety-critical  systems  are  important  in 
everyday  life  and  are  used  to  manage 
difficult  tasks  that  may  otherwise  be 
impossible  to  do.  Many  industries  use  sys¬ 
tems  that  pose  potential  hazards  to  the 
general  public,  such  as  systems  designed 
for  the  aviation  and  social  engineering 
industries.  When  systems  are  developed 
for  use  in  hazardous  situations,  develop¬ 
ment  strategies  and  ethics  standards  must 
change  to  fit  the  needs  of  the  project  in 
order  to  ensure  the  safety  of  employees 
and  the  general  public.  As  the  demand  for 
these  systems  continues  to  grow,  stan¬ 
dards  and  strategies  developed  to  keep 
safety-critical  software  safe  will  continue 
to  evolve. 

Case  Studies 

To  illustrate  certain  points,  incidents 
involving  failures  in  Union  Carbide  plants, 
the  Patriot  Missile  Defense  System,  Iran 
Air  Flight  655,  and  Therac-25  will  be  ana¬ 
lyzed. 

Union  Carbide 

Around  midnight  on  December  2,  1984, 
a  large  amount  of  water  was  introduced 
into  a  Methyl  Isocyanine  (MIC)  storage 
tank  at  Union  Carbide’s  Bhopal,  India 
pesticide  plant,  producing  one  of  the 
worst  industrial  accidents  in  history. 
Once  mixed  with  water,  the  MIC  pro¬ 
duced  a  vapor  that  escaped  through  the 
ventilation  system  into  the  atmosphere. 
Quickly  afterward,  the  MIC  began  to 
decompose  into  many  different  compo¬ 
nents,  including  cyanide  gas,  and  “...  the 
immediate  death  toll  from  the  cyanide 
release  was  in  excess  of  2,000  people” 
[1].  In  the  years  to  follow,  the  total  death 
and  injury  counts  ranged  in  the  hundreds 
of  thousands. 

Though  explanations  vary,  most 
attribute  the  disaster  to  a  combination  of 
human  error  and  faulty  safety  systems. 
The  system  also  lacked  software  safe¬ 
guards  that  would  prevent  water  mixture 
if  certain  hardware  safeguards  were  not 
in  place. 


Patriot  Missile  Defense  System 

In  1991,  a  software  design  error  in  the 
Patriot  Missile  Defense  System  caused  the 
deaths  of  28  US.  soldiers  when  a  missile 
was  fired  at  the  wrong  destination.  After 
investigation,  it  was  determined  the  error 
was  due  to  differences  in  the  clocks  of  the 
internal  missile  system  and  the  global  sys¬ 
tem,  resulting  in  a  miss  when  intercepting 
an  incoming  missile. 

Iran  Air  Flight  655 

Iran  Air  Flight  655,  a  civilian  air  flight 
traveling  on  an  approved  flight  path,  was 
shot  down  by  a  US.  guided  missile  cruiser 
on  July  3,  1988.  A  US.  Navy  investigation 
discovered  the  plane  was  shot  down 
because  of  both  a  bug  in  the  Advanced 
Electronic  Guidance  and  Instrumentation 
System  (AEGIS)  aboard  the  craft  along 
with  bad  decision-making  skills  by  the 
commanding  officer. 

At  no  time  did  the  AEGIS  system  fail 
completely;  the  order  to  fire  was  given  by 
the  commanding  officer  because  of  oper¬ 
ator  error  and  lack  of  judgment.  Once  the 
aircraft  was  identified  by  AEGIS,  the  sys¬ 
tem  then  proceeded  to  ask  the  aircraft  for 
its  friend  or  foe  (FOF)  status.  The  com¬ 
mercial  aircraft  then  responded  with  the 
code  commair,  identifying  itself  as  a  com¬ 
mercial  aircraft.  After  trying  to  confirm 
the  FOF  status  with  repeated  radio  com¬ 
munication,  warning  signals  were  given  to 
have  the  aircraft  change  course. 

Without  any  response  from  the  air¬ 
craft,  AEGIS  was  instructed  to  read  the 
FOF  signal  from  the  plane  again.  This 
posed  a  problem  as  AEGIS  could  not  read 
the  FOF  signal  from  the  same  plane  twice 
without  rebooting,  causing  the  system  to 
read  the  signal  of  a  fighter  jet  stationed  at 
a  nearby  air  base.  With  a  new  signal  indi¬ 
cating  a  military  aircraft,  the  commander 
issued  the  order  to  fire. 

Therac-25 

Between  June  1985  and  January  1987 
Therac-25,  a  Medical  Linear  Accelerator 
designed  by  Atomic  Energy  of  Canada 


Limited  (AECL),  malfunctioned  multiple 
times.  These  malfunctions  resulted  in  the 
deaths  of  three  patients,  with  many  more 
being  injured. 

Unlike  previous  inventions  by  AECL, 
the  Therac-25  did  not  use  proven  hard¬ 
ware  with  interlocks  to  ensure  safety. 
Instead,  Therac-25  relied  completely  on 
software  modified  from  older  systems  in 
order  to  reduce  costs  [2].  Because  of  this 
design  choice,  there  was  no  safety  proto¬ 
col  when  the  software  failed. 

Based  on  user  feedback,  AECL  modi¬ 
fied  the  software  before  the  initial  release 
to  include  a  quick  re-entry  method  that 
allowed  the  operator  to  skip  re-entry  of 
certain  treatment  information  [2].  This 
step  was  initially  used  to  ensure  the  patient 
was  getting  the  correct  dosage  prescribed. 
However,  the  change  was  not  thoroughly 
tested,  and  as  a  result  a  software  bug  was 
never  found. 

If  used  extremely  quickly,  the  system 
would  generate  an  error  and  subsequently 
notify  the  operator  of  a  cryptic  error  mes¬ 
sage.  After  resetting  the  device  to  clear  the 
error,  the  machine  would  not  indicate  any 
dosage  was  applied,  but  the  patient  would 
receive  a  hazardous  overdose  of  electrons. 

Identifying  Hazards 

Hazard  analysis,  though  important  in 
every  software  engineering  discipline,  has 
an  extremely  pronounced  role  when  deal¬ 
ing  with  safety-critical  systems.  There  are 
several  methods  to  perform  hazard  analy¬ 
sis,  but  before  analyzing  the  hazards  they 
must  be  identified. 

As  stated  in  [3],  “There  does  not 
appear  to  be  any  easy  way  to  identify  haz¬ 
ards  within  a  given  system.”  Because  of 
this  fact,  many  identification  methods  are 
often  used  in  order  to  find  as  many  haz¬ 
ards  as  possible  in  a  given  system.  The  pri¬ 
mary  techniques  used  in  this  process  are 
the  Delphi  Technique  and  joint  applica¬ 
tion  design  (JAD) . 

Delphi  Technique 

The  Delphi  Technique,  developed  by  the 


26  CrossTalk  The  Journal  of  Defense  Software  Engineering 


September  2008 


Hazardous  Software  Development 


Rand  Corporation  in  the  early  1980s,  is  an 
identification  technique  that  aims  at  gath¬ 
ering  different  ideas  from  different  geo¬ 
graphical  locations. 

The  advantages  of  this  technique  are 
numerous.  The  technique  requires 
anonymity,  reducing  the  chance  one  voice 
win  be  heard  over  the  opinions  of  others. 
This  technique  also  eliminates  the  need 
for  participants  to  congregate,  removing 
the  stress  of  arguments  between  partici¬ 
pants  and  the  struggle  of  a  group  reaching 
a  consensus. 

Even  with  its  advantages,  the  Delphi 
Technique  can  be  slow  to  administer. 
The  technique  uses  questionnaires,  poUs, 
and  other  traditional  information-gath¬ 
ering  techniques.  Because  of  this  limita¬ 
tion,  the  Delphi  Technique  calls  for  mul¬ 
tiple  iterations,  slowing  down  the 
process.  The  process  could  also  be 
slowed  by  the  amount  of  workload  on 
the  moderator. 

JAD 

JAD  is  a  hazard  brainstorming  system  pio¬ 
neered  by  IBM.  Generally,  JAD  meetings 
are  used  in  order  for  a  group  of  people  to 
reach  an  agreement  on  what  pq^es  of  haz¬ 
ards  could  occur  and  how  those  hazards 
could  affect  the  system  or  a  person’s  qual¬ 
ity  of  life  [3] . 

JAD  development,  though  fast  and 
thorough,  was  originally  designed  for 
other  uses  then  adapted  to  be  used  in  haz¬ 
ard  identification.  Because  of  this,  JAD 
does  not  seem  to  identify  hazards  as  well 
as  other  development  styles.  JAD  also 
forces  a  consensus  on  the  people  involved 
in  the  process,  which  is  often  difficult  to 
reach. 


Analyzing  Hazards 

Hazard  analysis  is  the  process  of  examin¬ 
ing  all  identified  hazards,  with  the  goal  of 
locating  the  points  of  failure  which  cause 
the  hazards.  There  are  two  different  types 
of  analysis:  inductive,  and  deductive  tech¬ 
niques.  Inductive  techniques  attempt  to 
start  with  answers  and  generate  the  ques¬ 
tions  during  the  analysis  process,  while 
deductive  techniques  look  for  answers  to 
questions  that  are  pre-defined.  This  article 
win  focus  only  on  inductive  techniques, 
though,  including  event  tree  analysis,  fail¬ 
ure  modes  and  effects  analysis,  hazard  and 
operability  analysis,  and  fault  tree  analysis. 

Fault  Tree  Analysis 

Fault  tree  analysis  focuses  on  one  hazard 
and  works  to  find  any  events  that  could 
trigger  the  hazard  to  occur.  The  hazard  in 
question  is  placed  at  the  top  of  the  tree 


Figure  1 :  A  Sampk  Vault  Tree  Analysis  Document 


with  each  subsequent  event  that  could 
trigger  the  main  hazard  underneath  it. 

Figure  1  shows  an  example  fault  tree 
diagram  of  a  car  where  a  vehicle  collision 
is  the  final  outcome.  The  diagram  identi¬ 
fies  many  points  of  failure,  such  as  the 
brakes  and  the  driver’s  ability.  While  driver 
ability  cannot  be  controlled  in  design,  the 
brake  design  can  now  be  prioritized  to 
ensure  safety. 

A  fault  tree  diagram  can  serve  many 
purposes,  the  most  important  being  the 
ability  to  determine  different  factors  to  a 
system  failure.  Fault  tree  diagrams  can  also 
be  used  during  all  the  phases  in  a  software 
life  cycle,  reducing  the  chances  of  missing 
a  hazard  after  the  planning  stages. 

The  main  downside  to  fault  tree  analy¬ 
sis  is  the  high  cost  of  producing  docu¬ 


mentation.  Therefore,  it  is  rarely  used  out¬ 
side  of  high-risk  situations  [3]. 

Event  Tree  Analysis 

Event  tree  analysis  has  many  similarities  to 
fault  tree  analysis,  but  instead  focuses  on 
finding  the  outcomes  of  a  system  failure. 
Figure  2  shows  an  example  of  an  event 
tree  diagram  for  a  car  brake  system.  This 
specific  example  looks  at  a  braking  hazard, 
with  the  main  brake  and  emergency  brakes 
being  the  points  of  failure.  Branching  at 
each  failure  point,  system  developers  can 
see  the  outcome  of  almost  every  failure 
situation. 

Hazard  and  Operability  Analysis 

Hazard  and  operability  analysis  (HOA),  or 
operating  hazard  analysis,  is  a  hazard 


Figure  2:  A  Sample  Event  Tree  Analysis  Document 
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analysis  technique  that  requires  attention 
during  aU  stages  of  the  software  engineer¬ 
ing  development  cycle.  By  using  HOA,  a 
project  manager  can  ensure  the  maximum 
number  of  hazards  are  identified  and  cor¬ 
rected  before  product  release  [3], 

HOA  uses  two  different  steps  in  order 
to  analyze  hazards.  First,  each  hazard  is 
either  classified  into  being  a  human  hazard 
or  an  environmental  hazard.  After  group¬ 
ing  the  hazards,  they  are  analyzed  further 
to  generate  a  list  of  problems  that  can 
result  from  each  specific  hazard.  Each 
point  in  the  process  is  then  checked  to  see 
if  it  is  a  safety-critical  concern.  If  deemed 
safety-critical,  the  hazard  is  analyzed  to 
determine  how  to  minimize  the  possibility 
of  its  occurrence. 

This  type  of  hazard  analysis  could 
have  been  used  for  the  system  at  Union 
Carbide.  By  searching  for  potential  haz¬ 
ards  throughout  every  step  of  the  design 
process,  the  development  team  may  have 
found  more  points  of  failure  in  the  system 
and  designed  safeguards  around  these  fail¬ 
ure  points. 

Failure  Modes  and  Effects  Analysis 

Failure  modes  and  effects  analysis  was 
first  developed  in  1 984  by  the  Department 
of  Defense  as  a  measure  to  prevent  haz¬ 
ardous  failures  of  critical  systems.  This 
method  uses  a  table  structure  to  take  every 
component  of  the  system  and  evaluate 
different  failure  modes  for  that  compo¬ 
nent.  This  type  of  analysis  also  is  used  to 
find  [3]: 

•  The  effect  of  component  failure. 

•  The  cause  of  component  failure. 

•  The  occurrence  chances. 

•  The  severity  of  the  problem. 

•  The  probability  of  detecting  the  prob¬ 
lem  before  the  hazardous  situation 

occurs. 

Figure  3  shows  an  example  of  a  fail¬ 
ure  modes  and  effects  document  based 
around  a  vehicle  tie  bar  bracket.  By  ana¬ 
lyzing  all  possible  failure  modes,  design¬ 
ers  of  the  tie  bar  bracket  can  see  that 
invalid  specifications  of  certain  parts 


could  cause  the  bracket  to  fail. 

Examples 

The  blame  for  Bhopal  was  primarily 
placed  on  the  shoulders  of  the  developers 
of  the  system,  not  on  the  operators  them¬ 
selves  [1].  Several  ways  are  listed  that  the 
development  team  could  have  been  better 
prepared  for  when  working  on  a  safety- 
critical  system.  Some  points  described  in 
[4]  include: 

•  Setting  learning  objectives  for  the 
developers,  such  as: 

°  Assigning  a  design  manager  to 
ensure  safety  precautions  are 
taken. 

°  Creating  a  safety  program  to 
oversee  the  project. 

°  Conducting  random  audits  of 
software. 

**When  a  safety-critical 
system  does  fail,  it 
then  becomes  the 
responsibility  of  the 
company  to  follow  up 
and  take  charge  of 
the  problem.** 

•  Making  sure  employees  are  receiving 
education  and  not  shrugging  off  their 
responsibilities. 

•  Learning  from  past  mistakes  of  system 
failures. 

•  Including  safety-critical  engineering  top¬ 
ics  into  software  engineering  courses. 

A  couple  reasons  why  safety  is  often  given 
second-place  status  to  other  factors  in 
design  is  provided  in  [5].  Because  safety 
issues  often  are  ignored  or  unknown, 
these  issues  generate  a  substantial  amount 
of  risk  to  product  success  and  personal 
safety.  These  are  risks  outlined  in  [3] 


which  must  be  taken  into  consideration 
when  doing  both  hazard  identification  and 
analysis.  Some  other  risks  include  the  fol¬ 
lowing: 

•  Developers  not  conforming  to  the 
safety  engineering  processes  due  to 
time  restraints. 

•  Developer  lack  of  safety-oriented 
training. 

•  Lack  of  automated  safety  shutdown 
procedures  in  an  emergency  situation. 

Conclusion 

Had  the  design  team  at  Bhopal  used  one 
of  these  development  strategies,  they 
could  have  been  more  prepared  for  the 
conditions  which  caused  the  1984  disaster. 
If  safety  education  and  safety-minded 
development  were  taken  into  account  ear¬ 
lier  in  the  design  process,  the  system  may 
have  prevented  the  operator  from  mixing 
the  solution  unless  the  certain  safety  con¬ 
ditions  were  met. 

Hazard  identification  and  analysis  is 
extremely  important  in  any  safety-critical 
system,  allowing  for  potential  hazards  to 
be  found  and  analyzed  before  they  cause 
problems  in  a  final  product. 

These  steps  also  go  hand-in-hand  with 
verification  and  vaUdafion  (V&V)  steps 
defined  by  the  Institute  of  Electrical  and 
Electronics  Engineers  (IEEE)  standards 
document  1012.  This  document  defines 
guidelines  on  how  to  first  identify  and 
manage  hazards  and  risks.  V&V  steps 
ensure  that  problems  are  documented  and 
then  followed  through,  resulting  in  safer 
systems. 

Codes  of  Ethics 

With  the  amount  of  safety-critical  systems 
currently  in  use,  ethics  play  a  larger  role  in 
software  engineering  today  than  they  ever 
have  in  the  past.  The  Computer  Society,  in 
cooperation  from  the  IEEE,  put  together 
a  set  of  ethics  codes  which  all  software 
engineers  and  designers  should  follow  in 
order  to  assure  software  quality  and  safety. 
The  standards  relating  to  hazardous  devel¬ 
opment  discussed  in  [4]  include,  but  are 
not  limited  to: 

•  Assuring  that  software  engineers 
working  on  a  project  take  responsibili¬ 
ty  for  the  code  they  write  or  decisions 
they  make.  For  example,  AECL  never 
revealed  the  developer  of  the  Therac- 
25  system;  therefore,  there  was  never  a 
person  to  question  about  the  software 
issues.  This  is  often  hard  to  enforce,  as 
the  original  engineer  may  not  be  part 
of  the  company  anymore. 

•  Approving  software  only  if  it  meets 
specifications  of  safety  and  perfor¬ 
mance,  passes  aU  tests  required,  and 


Figure  3:  A  Sample  Vailure  Modes  and  Effects  Analysis  Document  for  a  Tie  Bar  Bracket 
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does  not  hinder  the  life  or  safety  of  a 
person. 

•  Ensuring  that  any  end-user  documen¬ 
tation  for  the  software  is  fully  pre¬ 
pared.  This  was  also  a  problem  in  the 
Therac-25  system,  as  AECL  did  not 
include  definitions  of  their  error  mes¬ 
sages  inside  of  the  operator’s  manual. 
Because  of  this,  many  users  of  the  sys¬ 
tem  just  bypassed  errors  the  system 
displayed. 

•  Disclosing  any  risks  or  hazards  known 
about  the  system  prior  to  the  installa¬ 
tion  of  the  product. 

•  Cooperating  with  any  concerns  direct¬ 
ed  at  the  company  regarding  safety. 
This  includes  fully  investigating  any 
safety  claims.  In  the  case  of  AEGIS, 
the  company  quickly  changed  their 
systems  over  to  the  Linux  operating 
system  in  order  to  fix  rebooting  prob¬ 
lems.  Therac-25  did,  in  some  respects, 
cooperate  with  the  customers  but 
refused  to  take  any  extra  actions  until 
it  was  proved  that  their  product  was 
causing  problems. 

Besides  the  examples  mentioned. 
Union  Carbide  possibly  violated  ethical 
standards  by  not  adding  safety  functions 
to  their  software.  Problems  with  the  sys¬ 
tem,  though  probably  known  during 
development,  did  not  implement  any  safe¬ 
ty  interlocks  to  stop  chemical  mixings 
without  at  least  one  of  the  safety  imple¬ 
mentations  in  place.  AECL  also  violated 
ethical  standards,  even  going  so  far  as  con¬ 
tinuing  sales  of  Therac-25  units  after  mul¬ 
tiple  accounts  of  product  over  dosages. 
Iran  Air  Flight  655  was  shot  down  because 
the  developers  of  AEGIS  forced  reboots 
after  an  active  FOF  signal  was  sent  to  a 
plane.  Because  the  system  was  not  forced 
to  reboot,  this  could  constitute  as  an  ethi¬ 
cal  conflict. 

Aftermath  Management 

When  a  safety-critical  system  does  fail,  it 
then  becomes  the  responsibility  of  the 
company  to  follow  up  and  take  charge  of 
the  problem.  There  are  many  ways  to  deal 
with  the  problem;  one  of  them  being  a 
process  called  closed  loop  corrective 
action  (CLCA). 

CLCA 

CLCA  is  a  process  developed  by  the 
International  Organization  for  Standard¬ 
ization  (ISO)  standards  foundation  in 
order  to  make  sure  companies  respond 
correctly  to  reports  of  hazardous  situa¬ 
tions.  The  usage  of  the  CLCA  process 
may  either  be  triggered  by  ISO  9000  non- 
compliance  or  by  concerns  from  the  com¬ 
pany  itself  Depending  on  the  severity  of 


the  problem,  the  system  may  even  require 
a  complete  rewrite  of  the  working  build 
[6].  CLCA,  also  known  as  corrective  and 
preventive  action,  is  something  that  is 
planned  in  order  to  prevent  an  action 
from  happening  in  the  future.  Closed  loop 
just  means  the  development  process  is 
closed  until  problems  with  the  system  are 
corrected.  It  is  described  in  [6]  as: 

The  pattern  of  activities  which 
traces  the  symptoms  of  a  problem 
to  its  cause,  produces  solutions  for 
preventing  the  recurrence  of  the 
problem,  implements  the  changes 
and  monitors  that  the  changes  have 
been  successful. 

There  are  a  couple  of  steps  to  using 
CLCA: 

1.  Identify  the  problem. 

2.  Find  the  root  cause  of  the  problem. 

3.  Define  procedures  to  correct. 

4.  Implement  the  new  changes. 

5.  Follow  up  on  the  changes. 

Results 

CLCA  processes  could  have  helped  in 
both  the  Bhopal  and  Therac-25  incidents. 
Bhopal  could  have  recalled  their  entire 
tank  mixing  solutions  after  the  incident 
and  created  measures  that  prevented  the 
system  from  running  without  safety  mea¬ 
sures  in  place. 

Therac-25  could  have  used  some  sort 
of  CLCA  to  handle  some  of  the  concerns 
by  medical  professionals  after  the  system 
killed  three  people.  AECL  denied  respon¬ 
sibility  at  first  because  they  could  not 
reproduce  the  problem.  After  the  first 
scare,  AECL  did  not  do  extensive  testing 
on  each  claim,  stating  that  the  system  was 
completely  safe  [2]. 

Conclusion 

By  using  hazard  identification  and  analysis 
techniques,  many  of  the  problems  that 
plagued  Union  Carbide,  the  Patriot  Missile 
System,  AEGIS,  and  Therac-25  could 
have  been  avoided.  These  techniques 
allow  for  systems  engineers  and  software 
developers  to  find  and  remove  hazards 
and  risks  before  they  turn  into  liabilities. 
Along  with  these  techniques,  CLCA 
allows  for  companies  to  efficiently  deal 
with  system  failures  in  an  ethical  way. 
Using  these  preset  strategies  and  learning 
from  past  mistakes,  companies  can  avoid 
lawsuits  and  make  products  safer  for  con¬ 
sumers. ♦ 
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Seven  Words  You  Can  Never  Say  on  a  Flailing  Project 


Stand-up  comedy  is  a  short-lived  profession.  It  requires  quick 
wit  and  impeccable  timing  to  deliver  a  steady  stream  of  laughs 
to  a  short-fused  audience.  Comics  are  at  the  mercy  of  the  audi¬ 
ence,  defdy  playing  off  their  disposition  and  predilection.  Stand- 
up  comedy  is  definitely  not  for  the  faint  of  heart;  even  the  best 
comics  burn  out  or  decay.  Some,  like  Richard  Pryor,  Lenny 
Bruce,  and  John  Belushi  implode.  Others,  like  Eddie  Murphy, 
Roseanne  Barr,  and  Tim  Allen  go  stale.  The  smart  ones  transition 
to  more  secure  jobs  like  sitcoms  (Jerry  Seinfeld)  or  movies  (Steve 
Martin) .  Few  have  long,  productive  careers  in  stand-up. 

One  exception  is  George  Carlin,  who  passed  away  June  22  at 
age  71.  Carlin  had  a  remarkably  long  career  of  50-odd  years  in 
stand-up.  Sure,  he  had  dry  spells  as  Mr.  Conductor  and  the  nar¬ 
rator  of  Thomas  the  Tank  Engine;  nevertheless,  stand-up  was  his 
staple,  spanning  generations.  His  genius  was  making  you  think 
while  you  laugh;  for  example,  “...  if  crime  fighters  fight  crime  and 
fire  fighters  fight  fire,  what  do  freedom  fighters  fight?” 

Carlin  transformed  stand-up  comedy  by  introducing  a  sub¬ 
versive  approach  aimed  at  the  hypocrisy  in  our  daily  lives.  He 
produced  a  rich  body  of  work  covering  a  diverse  variety  of 
issues,  but  sadly  he  will  be  remembered  most  for  the  seven  words 
you  can  never  say  on  television. 

Project  managers,  especially  software  project  managers,  share 
similar  pressures  with  stand-up  comedians.  Good  software  pro¬ 
ject  management  requires  quick  wit,  impeccable  timing,  and  con¬ 
sistent  delivery  to  touchy  customers.  It  is  definitely  not  for  the 
faint  of  heart  or  half-hearted. 

Rather  than  tender  yet  another  list  of  software  best  practices,  I 
offer  an  elegy  to  projects  in  trouble  with  the  seven  words  you  can 
never  say  on  a  flailing  project.  You  never  get  that  list.  It’s  typically 
learned  by  trial  and  error,  but  I’ll  save  you  the  pain.  The  words  are: 
grit,  bUss,  instruct,  blunt,  customer,  milestone,  and  wits. 

Let’s  start  with  GRIT.  Never  say  grit  on  a  flailing  project 
unless  you  are  referring  to  abrasive  granules.  Avoid  the  grit  of 
indomitable  spirit  and  firmness  of  character  —  Rooster  Cogburn 
grit,  true  grit.  True  grit  implies  project  leaders  that  possess  the 
tenacity  to  inspire  and  the  courage  to  hold  accountable.  Settle  for 
cheerleaders,  task  masters,  or  bean  counters  —  but  never  true  grit. 

Never  associate  BLISS  with  a  flailing  project.  Bliss  is  an 
unusual  word  for  any  project.  Carlin  inquired,  “If  lawyers  are  dis¬ 
barred  and  clergymen  defrocked,  doesn’t  it  follow  that  electri¬ 
cians  can  be  delighted?”  You,  in  turn,  may  ask,  do  engineers  actu¬ 
ally  like  projects  that  offer  a  challenge  and  sense  of  accomplish¬ 
ment?  Yes,  indeed,  your  staff  became  engineers  because  they  like 
to  build  and  are  blissful  building  things  that  captivate,  fascinate, 
and  intrigue.  Nip  that  in  the  bud.  Introduce  a  slow  burn  of  inani¬ 
ty,  monotony,  and  mental  torpor  to  draw  oxygen  out  of  the  pro¬ 
ject,  avert  any  bUss,  and  leave  the  project  in  pure  engineering  hell. 

Never  utter  the  word  INSTRUCT  on  a  flailing  project. 
Autodidactic  managers  and  engineers  agree:  they  don’t  need  no 
stinkin’  instruction.  Managers  believe  instruction  wastes  time; 
time  is  money  and  money  is  short.  Engineers  see  instruction  as 
a  waste  of  time,  time  that  should  be  spent  with  thingamajigs, 
doohickeys,  and  doodads  that  intrigue  the  mind.  If  you 
instruct,  you  impart  knowledge.  Imparting  knowledge  leads  to 
mutual  understanding.  Mutual  understanding  could  lead  to 
direction,  purpose,  objectives,  common  processes,  and  collab¬ 
oration.  The  next  thing  you  know,  project  destruct  will  become 
project  construct  because  you  dared  to  instruct.  Keep  your 


head  in  the  sands  of  credulity. 

On  a  flailing  project,  you  can  never  be  BLUNT.  Blunt  will 
only  invite  the  four  horsemen  of  flourishing  projects:  realism, 
directness,  honesty,  and  frankness.  Imagine  the  shock  you  would 
impose  if  requirements  were  realistic  and  clear,  plans  direct  and 
uncomplicated,  budgets  honest  and  pragmatic,  and  project  com¬ 
munication  frank  and  straightforward.  Imagine  the  hurt  feelings 
and  bruised  egos.  Imagine  the  nerve-racking  decisions  and  ardu¬ 
ous  trade-offs.  For  the  love  of  political  correctness,  you  can’t 
allow  that  to  happen.  Play  it  safe.  Remain  idealistic,  apposite,  and 
equivocal. 

CUSTOMER?  What  customer?  Never  mention  the  customer 
on  a  flailing  project.  Involving  the  customer  will  complicate 
things  —  after  all,  they  can’t  even  congeal  requirements?  In  today’s 
society,  customers  should  drive  up,  order,  pick-up,  and  drive  off 
No  questions  asked.  Do  not  be  appeasers.  Leave  your  customers 
out  of  the  loop. 

If  you  find  yourself  on  a  flailing  project,  never  bring  up  the 
word  MILESTONE.  After  all,  a  milestone  is  a  stone,  stones 
weigh  you  down  and  milestones  weigh  you  down  with  account¬ 
ability.  Why  gild  the  lily  with  measures  and  meetings?  Who  wants 
to  know  where  your  project  really  stands?  Who  needs  course  cor¬ 
rections?  If  you  must  have  milestones,  choose  one  and  only  one; 
and  at  all  costs  avoid  the  milestone’s  progeny  —  the  inchstone. 

Finally,  never,  under  any  circumstance,  allude  to  WITS  on  a 
flailing  project.  Nothing  subjugates  a  project  faster  than  astute 
team  members  who  perceive  relationships  between  seemingly 
incongruous  or  disparate  concepts,  designs,  and  processes.  Top 
talent  is  expensive,  headstrong,  and  hard  to  control.  Stop  dream¬ 
ing  of  a  deus  ex  machina.  Save  money  with  desultory,  nebbish,  dila¬ 
tory  engineers.  They  will  validate  your  best  guesses  and  ensure 
your  worst  fears.  Follow  Carlin’s  advice,  “Never  underestimate 
the  power  of  stupid  people  in  large  groups.” 

I  apologize  to  managers  in  control  and  command  of  their 
software  projects.  This  article  offers  you  little.  With  time  on  your 
hands,  maybe  you  can  solve  Carlin’s  greatest  conundrum,  “When 
someone  asks  you,  ‘a  penny  for  your  thoughts’  and  you  put  your 
two  cents  in,  what  happens  to  the  other  penny?” 

— Gary  A.  Petersen 

Arrowpoint  Solutions,  Inc. 
gpetersen@arrowpoint.us 

Can  You  BACKTalk? 

Here  is  your  chance  to  make  your  point,  even  if  it  is  a  bit 
tongue-in-cheek,  without  your  boss  censoring  your  writing.  In 
addition  to  accepting  articles  that  relate  to  software  engineer¬ 
ing  for  publication  in  CROSSTALK,  we  also  accept  articles  for 
the  BackTalk  column.  BackTalk  articles  should  provide  a 
concise,  clever,  humorous,  and  insightful  perspective  on  the 
software  engineering  profession  or  industry  or  a  portion  of  it. 
Your  BackTalk  article  should  be  entertaining  and  clever  or 
original  in  concept,  design,  or  delivery.  The  length  should  not 
exceed  750  words. 

For  a  complete  author’s  packet  detailing  how  to  submit 
your  BackTalk  article,  visit  our  Web  site  at 
<www.stsc.hill.afmil>. 
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Homeland 

Security 


Software  Assurance 
Program 


Software  is  essential  to  enabling  the  nation’s  critical  infrastructure.  To 
ensure  the  integrity  of  that  infrastructure,  the  software  that  controls  and 
operates  It  must  be  reliable  and  secure. 


Visit  https://www.us-cert.gov/SwA  to  learn  more 
about  the  software  assurance 
program  and  how  you  can  @ 

become  more  involved. 


Security  must  be  “built  in” 
and  supported  throughout 
the  lifecycle. 


https://BuildSecurityln.us-cert.gov  to 
learn  more  about  the  practices  for 
developing  and  delivering  software  to 
provide  the  requisite  assurance. 

Sign  up  to  become  a  free  subscriber 
and  receive  notices  of  updates. 
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The  Department  of  Homeland  Security  provides  the  public-private  framework 
for  shifting  ilie  paradigm  from  “patch  management”  to  “software  assurance.” 
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